Darrin do you know of a way to validate coordinates for any given coordinate system? Users can import/enter data in any coordinate system so I am not sure how to validate that since the valid coordinate range is different for each projection.

We could convert everything to lat/long and validate the lat/long value.

This also applies to "bad" GPS points. Perhaps validation should also include Cybertracker sdata that includes too many days (more than 30?) or days with no data. This seems to happen with Cybertracker sometimes.

RRI Discussion:
What does validation mean? We could:
a) convert everything to lat/long and validate whether the values are between -180 and 180
b) validate that point is within the conservation area boundary - this requires a conservation area boundary to be set and I expect some points would correctly lie outside this boundary
c) validate the point is within a reasonable distance of the last point (what do with do with the first point)?
d) allow users to provide a polygon that all points must fall within
What to do with invalid point?
a) add a “is valid” property to waypoints and flag them as invalid
b) delete them from the system
c) make the user either delete or change them
Rich also alluded to validating the date/time associated with the waypoint. Apparently some cybertracker patrol waypoints have the wrong date/time. This is another set of requirements.

An option that would work, I think:

-Allow a polygon to define an area within which all points must fall. Perform this check when data are downloaded/input.
-When a patrol with invalid points is created, open a dialog that alerts the user, shows the patol track with the bad points highlighted and gives these options:
1. Ignore
2. Delete
3. Manually enter correct coordinates
4. Manually drag the point to the correct location on a map (the view track points window?)
5. Interpolate based on prior and subsequent points.

We envision creating an expandable QA module with area check to start; and patrol speed check could be added based on ticket 827

So this ticket now includes a patrol speed QA check, and ticket 827 has been closed.

Ticket #784 Plan
• Create a plugin QA Design. This involves creating an expandable QA module as separate plugins. This will create a programming infrastructure so we can easily add QA tasks into a framework that defines data sources (Patrols, Independent Incidents, Surveys) and what QA tasks should be performed on the data. Any new QA checks can implement the required features of this framework and then can be applied in a new software version. This will keep all the QA code centralized, organized and well defined.

• Part of the Module will provide actions for any point(s) that are flagged from any of the QA routines. Users will then have the choice to go through one point at a time, or select groups of points and then:
o Ignore
o Delete
o Manually enter correct coordinates, date or time values
o Manually drag the point to the correct location on a map
o Interpolate based on prior and subsequent points

• It will be possible to run QA routines manually anytime through a new menu item. Users will select one or more “types” (Patrol, Mission etc), one or more QA routines (area, speed etc) and a date range.

• We will have some sort of mechanism to flag this data for review if the data queue items are processing automatically and can’t be reviewed immediately.

• Points to Check - This system will be able to check both track and observation points.

• QA routine #1 - Area of Operations - flag anything outside a provided boundary
o The user can define the boundary 1 of 2 ways,
a) Upload a shapefile with 1 or more polygons in it or
b) Pick one or more polygons from the existing area layers

• QA routine #2 - Speed Check - flag point locations that would require unreasonable speeds to be correct
o Users will configuration the maximum speeds for each transportation type in the QA module settings. Any points with values over the maximum will be flagged.
o The code will calculate the implied speed for each point starting at point #2 based on distance between previous point and current one, divided by the time difference of the points, converted to km/hr.
o Any points with a speed above the range specified for the appropriate transportation type will be flagged as invalid.

We initially recommended a new plugin but I question whether this is valuable. We have a lot of plugins already and I don't think we need to add more (it adds complexity I don't think we need). Because this code is going to add limited stuff to the UI and users can ignore it if they want to, I would suggest we put this code in the core existing modules (patrol, survey, incident) but design it in such a way that a third party could write a plugin to add addition qa checks if desired (or if we want to release a new updatesite with new QA plugins).

Some initial design details:
https://docs.google.com/a/refractions.net/document/d/1qspB0BGESvAvTG_zpYANGtwxe1zqNhTOrhmMqCALPRM/edit?usp=sharing

ready for testing - have incorporated some feedback from Jeff

In 6.0.a5, if the Manual Data Validation routines window is open any new QA routines added will not be shown until the Manual Data Validation Window is closed and reopened.

you can use the "refresh list" link at the bottom right of the table

Until you made mention of the "refresh list" I didn't even know it existed. It works as expected and refreshed the list when a new routine is created.
Could it be moved to upper part of the window where it will be noticed?

fixed rev 6249

The refresh button moved to the right of the Select All / De-Select All and this is a more intuitive location that will help users find it. Testing done in 6.0.a7