In this unit we have focused on open-source tools that are designed to filter and process textfiles and streams of text. Notably we have not focused on generating or manipulating images or graphs. However this knowledge is very useful when visualising complex results or large datasets, as most contemporary software (particularly web-based software) producing images and graphs requires their data and commands to be presented as plain text.
The goal of this assignment is to assess your understanding of the use of the shell and open source tools to effectively report on and visualise data from a large dataset. You will be assessed on the clarity and quality of your shellscript(s) to examine and report on the data. While the efficiency of your shellscript will not be assessed, you should take care to avoid any excessive slow practices
Remember, it's considered good practice to include some comments in your shellscripts, to explain the author's design and logic. Include your name and student number in a comment near the top of each of your shellscripts.
The tasks
Perth's Public Transport Authority (PTA) provides public access to its scheduled times, stop locations, and route information from its webpage www.transperth.wa.gov.au/About/Spatial-Data-Access. You may download your own copy of the data (about 90MB) by clicking on the first link "By downloading the data you are agreeing to the terms of the License..."
The data is released as a collection of inter-related textfiles following the Google Transit Feed Specification (GTFS), which is also used by many other public transport companies, worldwide.
1.
Develop a shellscript which checks for and accepts five command-line arguments:
- the name of a directory containing a set of GTFS files, and
- a starting and an ending location (as latitude/longitude pairs, typically home and work),
and produces a simple HTML webpage describing the "best" public transport route that should be taken to travel between the locations. The starting time of the journey should be as close as possible to (and obviously, after) the time the shellscript is run, so that the traveller can load the HTML page onto their smartphone and then commence their journey (ideally, a version of your program would run on the smartphone).
The definition of the "best" route is up to you, but reasonable ones include minimal walking time, minimal waiting time, and minimal travelling time.
Some simplifying assumptions
o The pathway from starting location to destination will typically involve walking to the starting bus, train, or ferry stop, travelling on the bus....., and walking from the final stop to the destination.
o Assume that each walking segment (if any) is shorter than 1000 metres.
o There is no need to consider the day-of-the-week on which a particular service runs. Assume that all services run every day.
o In order to reduce the number of potential journeys that need searching, the actual travel on a bus, train, or ferry, must commence within one hour of leaving the starting location (home).
The shellscript's output, an HTML page, does not have to be fancy nor contain any embedded links. It should simply list the starting and stopping bus-stops, the route number, times when the traveller should get on and off the bus (or train...). Your chosen and reported route should consist of only one bus, train, or ferry journey. Your webpage should report an error if a single journey (one bus, one train, ...) cannot be found between source and destination.
1. Suggested steps for this task
o Consider how you would travel between the two locations if you were a dumb robot with unlimited time, patience, and energy.
o Download the GTFS dataset, and skim its documentation webpage. See what is in each textfile, determine which files will be required, and which files not required.
o Discuss the problem with a colleague (your project partner?) as to how you'd each attack the problem. Merge. Iterate.
o Find the bus, train, or ferry stops that are within 1000m (walking distance) of the starting location.
o Determine which bus, train, or ferry routes use each of those stops, today, after the journey's starting time.
o Do any of those bus, train, or ferry routes pass within 1000m (walking distance) of the destination? Which journey has the shortest total time?
2.
Embed a Google Map into your webpage, showing the locations and times of the starting and stopping bus-stops, and the required route numbers.
3. Extra credit
This task is optional, but you may wish to attempt it to recover any lost marks from the first two tasks. The maximum mark for the whole assignment remains capped at 30 marks.
Extend your shellscript so that the reported route may require more than one bus, train, or ferry journey. The route chosen and printed may now require one or more transfers between different, interconnecting bus, train, or ferry routes. Thus, the webpage should identify when a transfer is required. Don't forget to include the multiple journey information on your Google Map, as well.
To calculate the distance, in metres, between a pair of latitude/longitude coordinates, you'll need to employ the haversine formula [Wikipedia].
You may wish to perform the calculation by invoking a single program or, if using AWK, by calling an AWK function. Here's the code for each:
- haversine.c (which will require compiling - see comments in file), and
- haversine.awk (which should be embedded in a larger AWK script).