Validation of Lat-Long for Waypoints
Currently there is no validation of waypoints exceeding the range of possible values. This was discovered by setting tracks based on waypoints and the validation for that function caught the invalid coordinates.
Invalid coordinates can be imported via CSV or manually created.
Invalid coordinates can be imported via CSV or manually created.
Leave a comment
RRI Discussion:
What does validation mean? We could:
a) convert everything to lat/long and validate whether the values are between -180 and 180
b) validate that point is within the conservation area boundary - this requires a conservation area boundary to be set and I expect some points would correctly lie outside this boundary
c) validate the point is within a reasonable distance of the last point (what do with do with the first point)?
d) allow users to provide a polygon that all points must fall within
What to do with invalid point?
a) add a “is valid” property to waypoints and flag them as invalid
b) delete them from the system
c) make the user either delete or change them
Rich also alluded to validating the date/time associated with the waypoint. Apparently some cybertracker patrol waypoints have the wrong date/time. This is another set of requirements.
What does validation mean? We could:
a) convert everything to lat/long and validate whether the values are between -180 and 180
b) validate that point is within the conservation area boundary - this requires a conservation area boundary to be set and I expect some points would correctly lie outside this boundary
c) validate the point is within a reasonable distance of the last point (what do with do with the first point)?
d) allow users to provide a polygon that all points must fall within
What to do with invalid point?
a) add a “is valid” property to waypoints and flag them as invalid
b) delete them from the system
c) make the user either delete or change them
Rich also alluded to validating the date/time associated with the waypoint. Apparently some cybertracker patrol waypoints have the wrong date/time. This is another set of requirements.
An option that would work, I think:
-Allow a polygon to define an area within which all points must fall. Perform this check when data are downloaded/input.
-When a patrol with invalid points is created, open a dialog that alerts the user, shows the patol track with the bad points highlighted and gives these options:
1. Ignore
2. Delete
3. Manually enter correct coordinates
4. Manually drag the point to the correct location on a map (the view track points window?)
5. Interpolate based on prior and subsequent points.
-Allow a polygon to define an area within which all points must fall. Perform this check when data are downloaded/input.
-When a patrol with invalid points is created, open a dialog that alerts the user, shows the patol track with the bad points highlighted and gives these options:
1. Ignore
2. Delete
3. Manually enter correct coordinates
4. Manually drag the point to the correct location on a map (the view track points window?)
5. Interpolate based on prior and subsequent points.
Ticket #784 Plan
• Create a plugin QA Design. This involves creating an expandable QA module as separate plugins. This will create a programming infrastructure so we can easily add QA tasks into a framework that defines data sources (Patrols, Independent Incidents, Surveys) and what QA tasks should be performed on the data. Any new QA checks can implement the required features of this framework and then can be applied in a new software version. This will keep all the QA code centralized, organized and well defined.
• Part of the Module will provide actions for any point(s) that are flagged from any of the QA routines. Users will then have the choice to go through one point at a time, or select groups of points and then:
o Ignore
o Delete
o Manually enter correct coordinates, date or time values
o Manually drag the point to the correct location on a map
o Interpolate based on prior and subsequent points
• It will be possible to run QA routines manually anytime through a new menu item. Users will select one or more “types” (Patrol, Mission etc), one or more QA routines (area, speed etc) and a date range.
• We will have some sort of mechanism to flag this data for review if the data queue items are processing automatically and can’t be reviewed immediately.
• Points to Check - This system will be able to check both track and observation points.
• QA routine #1 - Area of Operations - flag anything outside a provided boundary
o The user can define the boundary 1 of 2 ways,
a) Upload a shapefile with 1 or more polygons in it or
b) Pick one or more polygons from the existing area layers
• QA routine #2 - Speed Check - flag point locations that would require unreasonable speeds to be correct
o Users will configuration the maximum speeds for each transportation type in the QA module settings. Any points with values over the maximum will be flagged.
o The code will calculate the implied speed for each point starting at point #2 based on distance between previous point and current one, divided by the time difference of the points, converted to km/hr.
o Any points with a speed above the range specified for the appropriate transportation type will be flagged as invalid.
• Create a plugin QA Design. This involves creating an expandable QA module as separate plugins. This will create a programming infrastructure so we can easily add QA tasks into a framework that defines data sources (Patrols, Independent Incidents, Surveys) and what QA tasks should be performed on the data. Any new QA checks can implement the required features of this framework and then can be applied in a new software version. This will keep all the QA code centralized, organized and well defined.
• Part of the Module will provide actions for any point(s) that are flagged from any of the QA routines. Users will then have the choice to go through one point at a time, or select groups of points and then:
o Ignore
o Delete
o Manually enter correct coordinates, date or time values
o Manually drag the point to the correct location on a map
o Interpolate based on prior and subsequent points
• It will be possible to run QA routines manually anytime through a new menu item. Users will select one or more “types” (Patrol, Mission etc), one or more QA routines (area, speed etc) and a date range.
• We will have some sort of mechanism to flag this data for review if the data queue items are processing automatically and can’t be reviewed immediately.
• Points to Check - This system will be able to check both track and observation points.
• QA routine #1 - Area of Operations - flag anything outside a provided boundary
o The user can define the boundary 1 of 2 ways,
a) Upload a shapefile with 1 or more polygons in it or
b) Pick one or more polygons from the existing area layers
• QA routine #2 - Speed Check - flag point locations that would require unreasonable speeds to be correct
o Users will configuration the maximum speeds for each transportation type in the QA module settings. Any points with values over the maximum will be flagged.
o The code will calculate the implied speed for each point starting at point #2 based on distance between previous point and current one, divided by the time difference of the points, converted to km/hr.
o Any points with a speed above the range specified for the appropriate transportation type will be flagged as invalid.
We initially recommended a new plugin but I question whether this is valuable. We have a lot of plugins already and I don't think we need to add more (it adds complexity I don't think we need). Because this code is going to add limited stuff to the UI and users can ignore it if they want to, I would suggest we put this code in the core existing modules (patrol, survey, incident) but design it in such a way that a third party could write a plugin to add addition qa checks if desired (or if we want to release a new updatesite with new QA plugins).