1. System we have so far

Various master files are stored in publicwhip/members in the Public Whip CVS repository, which is on SourceForge. The main files are all-members.xml which contains lists of offices of membership of parliament, and people.xml which joins them together as people.

2. Happenings we have to do something in response to

2.1. reshuffle (including restructure of the government, departmental changes)

2.2. ministerial resignation

When they update the web page containing the list of ministerial positions, we notice that and automatically update the database accordingly; this then appears on the site. This is fine, apart from we might want to add some overrides when they're a bit slow/incorrect. Also, their web page lists everything twice - once each department, then the people in it, then list of people and their positions. We parse the second for positions, the first for responsibilities. A couple of people were only added to the second recently, when they should have been there for ages. This leads to:

http://www.theyworkforyou.com/mp/alistair_darling/edinburgh_south_west

2.3. mp dies,resigns etc (bye election called)

2.4. bye election takes place

The all-members.xml is currently semi-manually updated.

When case 3/4 bye election (or party change, or death etc.) happens, David Boothroyd's website http://www.election.demon.co.uk/strengths.html is updated to say so.
scripts/dailyupdate then emails fawkes@very.unfortu.net (which is sent to various people via .forward) to tell them this. The emails begin "Files strengths.html and strengths-last.html differ".
You manually fix up members/all-members.xml, and commit to ukparse SVN.
Rerun personsets.py in publicwhip/members. TODO: Automate this.
Some of the info converter scripts need rerunning, do this on cake: Run (locations need to be changed) publicwhip/members/edmconv.py. Run publicwhip/members/bbcconv.py. Commit the XML files changed by that lot. Then run ~/fawkes/scripts/mpinfoin.pl (to get new info in the database). TODO: Work out if there are any others that need running. Automate these steps somehow.
Optionally (if you're impatient like me), run "~/fawkes/scripts/xml2db.pl --all --members" to get the new member in the database immediately. Otherwise it will wait until next cron, when the change will be emailed as a check.
Both TheyWorkForYou.com and PublicWhip should then automatically pick up the new files, and do everything fine.
If somebody changed party (or any other change which is sudden, rather than with a gap that you always get for a byelection), make sure the parser didn't run and get the old MP id for intervening dates. The easiest way is probably to force a reparse of everything (debates and wrans) from the date of change on. TODO: This can be automated. Work out how to.

2.5. general election called (parliament dissolved)

2.6. general election occurs

6 is 659 cases of type 4. I think this must be a web form so we can make the chnages easily (and on election night, we can hunker down and get drunk while the results come in, at my place) - stef

After a general election, my plan was to get hold of the internal parliament list of MPs and run a script to convert that from HTML to the XML format. You have some time after the election before anything happens in Parliament. They all have to swear in for a few days and stuff. - Francis

2.7. 7 boundary changes (does anyone know if there are any due)

There is a constuencies.xml file, which does have constituency ids in them. I've never used the ids though, instead I have a "canonical name" for each constituency. Boundary changes will happen soon, but I think for the general election after the next one. It poses no especial problems - updating the postcode database needed. - Francis

2.8. 8 hansard do their yearly URL shuffle (late Feb / early March)

Yearly URL shuffle 8 is dealt with by the scraper script which notices the URLs have changed and will remap everything and change just those bits of the database. Have to make sure it works though next year, might be something minor missing. - Francis

Immutable page [current page size 4221 bytes] (last edited 2005-07-15 10:13:05 by glfd-cache-9.server.ntli.net)
FindPage by browsing, title search , text search or an index
Or try one of these actions: AttachFile, LikePages, LocalSiteMap, SpellCheck