Skip to content
jpmckinney edited this page Jul 22, 2012 · 7 revisions

List all the Rake tasks:

bundle exec rake -T

It seems the vast majority of Rake tasks are run once, and never again.

Run by a Capistrano task

rake run_stale_scrapers                               # Finds stale scrapers and runs them

Run by Whenever

rake cache_and_transfer_spending_data_summaries       # Cache spending data in YAML file and transfer to webserver
rake export_csv_planning_applications                 # Export CSV version of planning application data
rake export_csv_spending_data                         # Export CSV version of spending data
rake update_feed_entries                              # Update FeedEntries if not already running

Commented out:

rake run_stale_scrapers                               # Finds stale scrapers and runs them
rake import_council_officers                          # Import Council Officers from CLG CSV file

Defined in...

charities_scrapers

rake populate_charities                               # Populate UK Charities
rake get_missing_charities                            # Get Missing Charities
rake catch_up_with_new_charities                      # Catch up newly created charities
rake import_charity_annual_reports                    # Import Charity annual_reports
rake import_charity_class_types                       # Import Charity Classification Types
rake import_charity_classifications                   # Import Charity classification associations
rake import_charity_details                           # Import Data from Charity details table
rake import_charity_table                             # Import Data from Charity Register table
rake import_icnpo_class_types                         # Import ICNPO classification types
rake import_icnpo_classifications                     # Import ICNPO classifications for UK charities
rake import_1010_charities                            # Import 1010 charities

council_scraper

rake scrape_egr_for_councils                          # Quick and dirty scraper to get basic info about councils from eGR page
rake scrape_wdtk_for_names                            # Scrape WhatDoTheyKnow.com to get WDTK name
rake scrape_councils_for_feeds                        # Scraper council urls to get feed_url from auto discovery tag
rake import_ward_snac_ids                             # Import ONS SNAC codes into Wards table
rake enter_missing_snac_ids                           # Enter missing Ward SNAC ids
rake import_council_officers                          # Import Council Officers from CLG CSV file
rake enter_missing_ldg_ids                            # Enter missing Council LDG ids
rake import_os_ids                                    # Import OS Ids from SPARQL endpoint
rake import_country_districts_relationships           # Import County-District relationship from SPARQL endpoint
rake add_council_twitter_ids                          # add council twitter ids
rake export_sameas_relationships                      # export sameAs relationships
rake geocode_councils                                 # geocode council offices
rake import_council_cipfa_codes                       # Import CIPFA codes for councils
rake import_council_regions                           # Import council regions
rake match_1010_councils                              # Match 1010 councils
rake import_1010_councils                             # Import 1010 councils
rake populate_pension_funds                           # Populate pension funds
rake associate_pension_funds                          # Associate councils and pension funds
rake import_ons_gss_codes                             # Import ONS GSS codes into councils, wards tables
rake import_missing_council_latlongs                  # Import missing council lat longs
rake scrape_annual_audit_letters                      # Scrape annual audit letters
rake import_os_county_division_ids                    # Import OS Ids for County Electoral Divisions
rake import_missing_os_ids                            # Import missing OS IDs
rake import_wdtk_ids                                  # Import WDTK ids
rake update_1010_councils                             # Update 1010 councils
rake import_lgsl_ids                                  # Import latest Local Government Service List
rake import_missing_os_council_ids                    # Import missing OS council ids
rake import_os_parish_councils                        # Import parish councils from OS
rake import_nalc_parish_councils                      # Import NALC parish councils

councillors_utilities

rake import_councillor_twitter_ids                    # Import councillor twitter ids
rake scrape_sbe_cases                                 # Scrape Standards Board For England Cases
rake scrape_lgo_cases                                 # Scrape Local Government Ombudsman Cases

election_utilities

rake import_london_2006_election_results              # Import May 2006 London Election Results
rake import_london_2006_voting_proportions            # Import May 2006 London Voting Proportions
rake scrape_electoral_commission_party_list           # Scrape Electoral Commission for Political Parties
rake get_details_for_electoral_commission_parties     # Get details for Political Parties on Electoral Commission list
rake match_candidacies_and_members                    # Match candidacies and members
rake search_wikipedia_for_parties                     # Search Wikipedia For Political Parties
rake import_london_2002_election_results              # Import May 2002 London Election Results

geo_utilities

rake get_bounding_boxes_for_wards                     # Get bounding boxes for wards
rake import_postcodes_from_csv                        # Import postcodes from CSV file
rake import_ward_boundaries                           # Import Ward Boundaries

load_scrapers_and_parsers

rake planning_alerts:load_item_parser                 # Load item parser
rake planning_alerts:load_info_parser                 # Load info parser
rake planning_alerts:show_config                      # Show config
  • Other tasks have planning_alerts:load_config as a dependency.
  • planning_alerts:dump_portal_system is commented out.

planning_applications

rake export_csv_planning_applications                 # Export CSV version of planning application data
rake convert_caps_applications_to_idox                # Update CAPS planning application URLs to Idox ones

police_scrapers

rake populate_police_officers                         # Populate Police Officers from NPIA api
rake populate_police_team_info                        # Populate Police Teams from NPIA api
rake populate_police_teams                            # Populate Police Teams from NPIA api
rake populate_npia_ids                                # Populate NPIA ids for police forces
rake import_met_police_neighbourhoods                 # Scrape Met Police for neighbourhoods
rake scrape_police_forces                             # Scrape police forces
rake connect_police_force_to_la                       # Connect PoliceForce to LA
rake get_police_force_wikipedia_info                  # Get wikipedia info for Police Forces
rake get_police_force_wdtk_ids                        # Get WDTK ids for Police Forces
rake get_police_authority_info                        # Get info for Police Athorities
rake get_police_authority_wdtk_ids                    # Get WDTK ids for Police Authorities
rake connect_wards_and_police_teams                   # Connect Wards and Police Teams
rake populate_crime_types                             # Populate Crime Types from NPIA api
rake populate_crime_areas                             # Populate Crime Areas from NPIA api
rake populate_crime_area_info                         # Populate Crime Areas info from NPIA api
rake connect_postcodes_and_crime_areas                # Connect postcode and crime areas

resque

rake resque:move_queued_items                         # move queued items to new queue
rake resque:move_to_back_of_queue                     # move items to back of queue
rake resque:start_daemons                             # start all background resque daemons
rake resque:stop_daemons                              # stop all background resque daemons
rake resque:truncate_queue                            # truncate queue
rake resque:unregister_workers                        # unregister all workers
  • resque:clear_performer_locks is commented out.

spending_utilities

rake import_windsor_and_maidenhead_supplier_payments  # Import Windsor & Maidenhead Supplier Payments
rake old_import_gla_supplier_payments                 # OLD Import GLA Supplier Payments
rake match_suppliers_to_companies                     # Match suppliers to companies
rake move_supplier_info_to_company                    # Move supplier info to company model
rake import_richmond_supplier_payments                # Import LB Richmond Supplier Payments
rake import_ndpbs                                     # Import NDPBs
rake import_surrey_supplier_payments                  # Import Surrey CC Supplier Payments
rake import_uttlesford_supplier_payments              # Import Uttlesford Supplier Payments
rake import_kl_and_wn_payments                        # Import King's Lynn & West Norfolk Payments
rake import_barnet_payments                           # Import Barnet Payments
rake export_csv_spending_data                         # Export CSV version of spending data
rake import_islington_payments                        # Import Islington Payments
rake import_bedford_payments                          # Import Bedford Payments
rake import_bromley_payments                          # Import Bromley Payments
rake import_kandc_payments                            # Import Kensington & Chelsea Payments
rake import_spotlight_on_spend_data                   # Import Spotlight on Spend data
rake import_devon_cc_payments                         # Import Devon County Council Payments
rake import_lewes_payments                            # Import Lewes Payments
rake import_wigan_payments                            # Import Wigan Payments
rake import_wandsworth_supplier_payments              # Import Wandsworth Supplier Payments
rake import_corby_payments                            # Import Corby Payments
rake import_gla_payments                              # Import GLA Payments
rake import_south_glocs_payments                      # Import South Glocs Payments
rake import_hillingdon_payments                       # Import Hillingdon Payments
rake import_broxbourne_payments                       # Import Broxbourne Payments
rake import_west_dorset_payments                      # Import West Dorset Payments
rake import_waltham_forest_payments                   # Import Waltham Forest Payments
rake import_north_somerset_payments                   # Import North Somerset Payments
rake import_harlow_payments                           # Import Harlow Payments
rake import_cambridgeshire_payments                   # Import Cambridgeshire County Council Payments
rake import_lottery_grants                            # Import Lottery Grants
rake import_trafford_payments                         # Import Trafford Council Payments
rake import_birmingham_payments                       # Import Birmingham Payments
rake import_arts_council_grants                       # Import Arts Council Grants
rake import_transition_fund_grants                    # Import Transition Fund Grants
rake import_gla_suppliers                             # Import GLA Supppliers
rake cache_and_transfer_spending_data_summaries       # Cache spending data in YAML file and transfer to webserver
  • import_walsall_payments is commented out.

statistics_utilities

rake get_ness_subject_and_datasets                    # Get Ness Subjects and datasets
rake get_ness_ids                                     # Get Ness Ids for Councils
rake get_ness_dataset_topics                          # Get Ness Dataset topics
rake import_local_spending                            # Import Local Spending 2006-2007
rake add_dataset_relationships                        # Add Dataset relationships
rake convert_ness_selected_topics_to_groupings        # Convert NessSelectedTopics top topic_groupings
rake get_ness_selected_topic_info_for_councils        # get NessSelectedTopic info for councils
rake get_bounding_boxes_for_councils                  # get Bounding Boxes for councils
rake get_descriptions_for_ness_topics                 # get descriptions for Ness topics
rake import_socitm_bc10                               # Import SOCITM Better Connected Report 2010
rake import_oac_list                                  # Import Output Area Classification List
rake import_council_oacs                              # Import Council Output Area Classifications
rake import_ward_oacs                                 # Import Ward Output Area Classifications

temp_migration_tasks

rake create_document_precis                           # Create precis from document bodies
rake import_proclass                                  # Import Proclass classification
rake import_cpid_entities                             # Import CPID entities
rake import_planning_alerts                           # Import Planning Alerts
rake setup_caps_scrapers                              # Set up CAPS PlanningApplication scrapers
rake add_caps_info_scrapers                           # Add up CAPS InfoScrapers
rake add_fastweb_portal_system                        # Add FastWEB portal system
rake add_fastweb_item_parser                          # Create FastWEB item parser
rake add_fastweb_item_scrapers                        # Create FastWEB item scrapers for all councils
rake add_fastweb_info_parser                          # Create FastWEB info parser
rake add_fastweb_info_scrapers                        # Create FastWEB info scrapers for all councils
rake date_formats                                     # Show the date/time format strings defined and example output
rake import_last_four_years_planning_applications     # Import last four years planning applications for Idox scrapers
rake remove_duplicate_planning_applications
rake convert_old_confirmed_planning_alert_subscribers
rake convert_caps_urls_to_idox_urls
  • import_last_four_years_pas_for_lichfield is commented out.

Defined by...

delayed_job

rake jobs:clear                                       # Clear the delayed_job queue.
rake jobs:work                                        # Start a delayed_job worker.

rails_indexes

rake db:show_me_ar_find_indexes                       # collect indexes based on AR::Base.find calls.
rake db:show_me_some_indexes                          # scan for possible required indexes

sitemap_generator

rake sitemap:clean                                    # Delete all Sitemap files in public/ directory
rake sitemap:create                                   # Generate sitemaps but don't ping search engines.
rake sitemap:install                                  # Install a default config/sitemap.rb file
rake sitemap:refresh                                  # Generate sitemaps and ping search engines.
rake sitemap:refresh:no_ping                          # Generate sitemaps but don't ping search engines.