GPS Data Wish List

[FredHiltz]

Jennifer asked in the topic just below this for our thoughts on improving the GPS data that Adventure Cycling publish. That thread grew rather large and unwieldy, so I have started this wish list with several points from that thread.

Please take a look at Jennifer's introductory post to see what she is looking for, then comment on these points or add more.

In the order of their appearance:

1. Don't change too much.

2. Start and end the sample routes at the edges of maps where possible.

3. Include contact information in the waypoints marking points of interest.

4. Include routepoints for follow-the-road navigation by GPSRs that do not calculate this on their own.

5. Put the waypoints that mark points of interest and those that mark the riding route in separate files. Include routepoints in the latter file.

Fred

[mdxix]

I vote for 5 & 6. In my view, it is essential to keep the data sets independent, modular, and compatible with many platform.

I thought the blog that Jennifer wrote was very promising. She made statements about "robust database", "mobile mapping solution is not if, but when", & "waypoints can be kept up to date easily and in an unobtrusive way". I could not agree more.

On the several trips that I have taken on four ACA routes, I never used the files as is. Nor have I read in this forum that someone simply loaded the files as is in their GPS unit, regardless of make and model.

That is, the more these files are easy to handle, manipulate, and use on various tools the better. Hence, I add:

6. Keep open & accessible across multiple platforms.

In other words, I would rather there not be a preference toward a few models versus others. People are using all kinds of technology that is advancing every year, including mobile.

Instead, I prefer to see the files use open standards that are modular working on multiple platforms. Create, for example, a quality check list that includes:

• Test conversion to multiple formats and various online mapping tools (edited 24aug2012)
• Test on multiple GPS units (recruit some test volunteers from the membership community)
• Test the file on a mobile device (test volunteers can help)

This applies to both routes & waypoint—but keep them separate

Best of luck to your team. This is a big effort. Thank you for your time.

[tsteven4]

Here are some nits I would wish for assuming that the data continues to be distributed in gpx format:

7. Add elevations (gpx ele elements) for all waypoints (gpx wpt elements) and routepoints (gpx rtept elements, but not necessarily gpx extensions rpt elements).  Today I toss all the ele elements and fetch them from http://gisdata.usgs.net or if that fails from http://ws.geonames.org and build a SQL database of elevations.  This gives me elevations for all points, but the validity is problematic on bridges and tunnels as they fetched points will be on the surface, which in these cases is not where the route is.  The elevation data is useful in displaying profiles, although because of the sparsity of the gpx rtepts they are rough.  If your software cannot fetch the elevation data I could help with software but in my experience this is non-trivial and requires some maintenance as the web servers and protocols change with time.
8. Delete all times, i.e. (gpx time elements).  I don't think these are meaningful for the ACA data and they can cause difficulties when viewing the routes in some tools, e.g. google earth.  I could provide software to this if you don't want to take care of it upstream of the gpx file creation.
9. Delete all gpx extension Depth elements.  There are only two in the entire set today, and they are used incorrectly.   I could provide software to this if you don't want to take care of it upstream of the gpx file creation.
10. Validate all distributed gpx files.  While I haven't found invalid gpx files from ACA, invalid files are a common cause of issues reported on the gpsbabel mailing list. I can supply software to do this.  Note that this step doesn't change anything, it just guarantees that the gpx files that are about to be distributed obey the rules for gpx files as defined in the appropriate XML schema documents (xsd files).
11.  Consistently organize and name the folders and files for each route, including case and blanks in directory and file names.  These makes automated processing easier.

And one that I consider more important:

12. Have contiguous route segments, gpx rte elements, share a route point (gpx rtept).  This make display of the overall route appear as one, instead of a bunch of separate and disconnected segments.  Accidental violations of this are the most common reason I write Jennifer.

And one major request:

13. Continue to provide public access to all the gpx data, including the points of interest and route data.  There was an article in one of last years magazines that stated that points of interest data would be restricted to members in the future.  To me it seems restricting access to the points of interest is in conflict with the ACAs mission "Adventure Cycling Association inspires and empowers people to travel by bicycle."  I am concerned that a revised policy on the waypoint data might prevent me from providing online viewing tools and google earth files to the public, and I certainly believe the maps at http://tsteven4.qwestoffice.net/ inspire and empower people to travel by bicycle. 

I question 5, the separation of the point of interest (wpt) and route (rtept and rpt) data.  I see this as a violation of 6, "keep open and accessible across multiple platforms".  I think the separation is convenient for some applications, e.g. mdxix seems to want this, but I am not convinced this is generally desirable.  I would prefer they remain together.

I am still unsure of what Jennifer meant by "Do you like/use the sample routes provided?".  Can you cite an example of a sample route?  In 3 Fred seems to imply a sample route just corresponds to the route as shown on a paper map panel.

Thanks for all your work and including us in the conversation.

[FredHiltz]

... assuming that the data continues to be distributed in gpx format

This is a given. We have no intention to abandon GPX. When the project began, we distributed two other formats as well, attempting to cover the most popular of the many proprietary GPS data programs. Now we have standards, and the wonderful thing about standards is that there are so many of them. For practical production reasons, GPX and its Garmin extensions are likely to remain our mainstay. KML? Maybe, if enough members want it for bicycle navigation, this would be worth spending staff time (=money) on.

11.  Consistently organize and name the folders and files for each route, including case and blanks in directory and file names.

The File names section of the GPS Data User Guide describes our convention for naming the GPX files. If you find some that do not follow it, please let us know of the oversight. If we implement WL #5, we would add some indicator of routes vs. services.

The distributed data contain no folders. A single Zip file contains all the data for one ACA route.

12. Have contiguous route segments, gpx rte elements, share a route point (gpx rtept).  This make display of the overall route appear as one, instead of a bunch of separate and disconnected segments.  Accidental violations of this are the most common reason I write Jennifer.

You are right, those are accidental. We intend to make the last waypoint defining a GPS route and the first waypoint defining the next be the same.

I am still unsure of what Jennifer meant by "Do you like/use the sample routes provided?".  Can you cite an example of a sample route?  In 3 Fred seems to imply a sample route just corresponds to the route as shown on a paper map panel.

From the GPS Data User Guide: "We include GPS routes in the published file to make the ACA route stand out on the map. They might also serve as sample routes to help you create the actual routes for a day’s riding." We have built these from 25 - 30 waypoints in order to fit as many as possible into limited GPSRs. WL #2 would improve on that.

Fred

[tsteven4]

For practical production reasons, GPX and its Garmin extensions are likely to remain our mainstay.

GPX is great.

The distributed data contain no folders. A single Zip file contains all the data for one ACA route.

I was referring to the files and folders within the zipfile for each route.
Some examples of small inconsistencies are:
1) the sierra cascades gpx files are in the root directory of the zipfile, while those for all others are in a sub-directory,
2) the sub directories are usually named something like "UC01v004 Folder", but there are two cases where the directory name does not include " Folder", these are AM01v002/AM01v002.gpx and GM01v003/GM01v003.gpx
3) Sometimes Folder uses an upper case F and sometimes a lower case f.
4) usually the gpx files use a lower case v, but the following use an upper case V instead:
GD01v009 Folder/GD01V009.gpx   
GD02v009 Folder/GD02V009.gpx   
GD03v009 Folder/GD03V009.gpx   
GD04v009 Folder/GD04V009.gpx   
GD05v009 Folder/GD05V009.gpx   
GD06v009 Folder/GD06V009.gpx   
WE01v008 Folder/WE01V008.gpx
4) the pdf files mix the case of the v as well.
5) the great divide pdf file has an extra digit, GDAboutV0011.pdf
I will grant you that all of these issues are nearly insignificant, but consistency would make automated processing of the data easier.

[FredHiltz]

I don't know what extraction program is showing you this, but those folder names are part of the path to the file on Jenn's Mac. The folders exist only on her computer, not in the Zip file. This listing from WinZip shows what is in the file.

Yes, we can be more diligent about following our own naming style conventions. Until they are published next, your program can convert all those file names to lower case without causing any name conflicts.

Fred

[tsteven4]

The folders exist only on her computer, not in the Zip file.

Fred, the directory structure, i.e. the path or folder information, is saved in the zip file.  Look at the Path column in your WinZip listing.   Usually this directory structure is recreated by default when the file is unzipped.  I can continue to work around it if necessary, it isn't a big deal.

[FredHiltz]

Sort of. The name of the folder that held the file on Jenn's computer is stored as the path in the Zip file. Nothing else about that folder--not its size, not the other files that were in it, not its creation date--nothing else is in the Zip file.

Does your zip program not let you just extract the files? Get WinZip, which presents the option to rebuild the path with empty folders above the files or not.

Fred

[chrisch]

I would consider two more things in the future management of your GPS data.  My thoughts are specific to waypoints and not routing information:

First, consider adding an API to query the data.  For example, give me all campsites within 30km of a given location, or give me all the restaurants along the Northern Tier Route.  This would open the doors for the intelligent generation of waypoint lists that are specific to a users needs.  They can download the data that interests them the most and then upload it to their device.  Furthermore, such an API would allow third party developers to write smartphone applications that query and display the data.  This would be a great asset to those without GPS devices.

Second, consider taking a wiki approach to the data and allow users to rate, comment, and flag waypoints.  A collaborative approach would allow the best data to trickle to the top while dated or inaccurate data would be removed.

Good luck with your redesign!
Chris

[mdxix]

consider taking a wiki approach to the data and allow users to rate, comment, and flag waypoints.

I like that Chris, something along the lines of waymarking.com or some other location based services. Download the the desired points to the GPS, view on the web, or view on a mobile device. Brilliant.

[mdxix]

KML? Maybe, if enough members want it for bicycle navigation, this would be worth spending staff time (=money) on.

The request here is for open & accessible format. For those interested in KML, there are plenty of tools out there that would convert from GPX—no need to spend any of ACA time or money. That will only work if the GPX file is structured properly.

KML is just one example. There are many other formats that users may be interested in.

I will edit the original post to state this intent of my comment: Test conversion to multiple formats and various online mapping tools

[gubbool]

I am working on the route(s) for my next epic trip.  I used the files available in early 2010 for a 207 day, 10,000 mile, 27 state, double crossing of the US pretty much as it was downloaded (just stripping out side trails and alternates) - I found the routing to be very good; the only problem with the data was that my GPS was unable to figure out how to auto-route bike trails.  The waypoint data was used somewhat, but my gps unit knows where to find cheap food, Walmarts for tires and tubes, and ATM's.  I used routes from the ST, AC, TA, LC, NT, SC, WE, GC, ST-again, and Natchez Trace (which is part something).   


First, a suggestion/tip: Keeping waypoints along a route separate from the intermediate points within a route make it very easy to load them as POI's - for conversation, POI's do not consume memory resources like a waypoint.  Generally, the number POI's are unlimited, while newer gps's support >2000 waypoint, Garmin's newest is limited to 4,000.  The current version of my 2013 trip of about 8,000 miles has 9944 waypoints and 264 routes.  New units support only 200 routes so I will be grouping a few of those together.  [And yes, I know how to handle gps data for long trip on an old gps'r.  I paddled around Florida last summer - no routing there, just 1515 miles of waypoints.]

And second - find a decent way to handle trails.  I am planning to ride the GDMBR this spring and Garmin's Mapsource and my gps devices go crazy trying to auto-route the backwoods. Currently, where there is a big problem, I am striping the too few waypoints from routes and deleting the route itself,  This has led to some trust issues.  For example: Waypoint M01B90 at N46.16523 W112.37158 on Google Satellite and auto-routing of that section.  For the GDBMR trail I have the 20 4000-pt tracks from (topofusion) Scott Morris' trip and his related work and do not have any issue there, I will follow the tracks.  However, as I found while bicycling home from Maine in '08 and during my '10 trip, rails-to-trails and local bike paths are difficult to locate/determine when the gps unit finds a nearby road.

Warren
findingwarren.com

[FredHiltz]

Two excellent points, Warren. Thanks for writing.

...
First... Keeping waypoints along a route separate from the intermediate points within a route make it very easy to load them as POI's...

You ought to be able to do this now, using the waypoints' symbols. All the waypoints that lie on the route (that define the route) use the generic waypoint symbol. Lodging, stores, campgrounds, museums have different symbols.

MapSource can sort the waypoint list by symbol. I do not know BaseCamp well enough to say. You can then load the waypoints you want into the GPSR or into a .gpx file.

...
And second - find a decent way to handle trails.  I am planning to ride the GDMBR this spring and Garmin's Mapsource and my gps devices go crazy trying to auto-route the backwoods...

I'm afraid we are at the mercy of the mapmakers for this one. I notice Garmin's TOPO 24 supplies auto-route data for a few biking, hiking, and rail conversion trails. When they see enough market, they will probably do more trails on maps better suited to cycling.

Meanwhle, I switch to straight-line navigation in these situations. We wrote the ACA data with this expectation. It works pretty well on all but the most contorted trails.

Fred

[newfydog]

For the Great Divide, just go to bikepacker or topofusion and find the most recent tracklog.  No need to autoroute or way points when you have a really detailed tracklog.

For a base, go to GPSfiledopot and download the state topomaps.

You will know if you are off route by 100 feet. 

Send me a pm and I'll give you good links and stuff, I'm suffering on a beach in Mexico at the moment.