Still Struggling With Localize

[Jim Frame]

It is not clear for me what goal for that workflow, when initially you use “rough (within a foot or two) search locations for the desired points“ as actual design points.

This is actually a common situation for me. A typical example is when I need to survey a parcel of land for which I have deed dimensions. In CAD I recreate the parcel configuration from the deed, then overlay that on a georeferenced aerial image, matching the deed outline to visible features (e.g. fences, roads) as best I can. Then I create points at the parcel corners so I can get close enough to search for monuments in the field. That's what I mean by rough search coordinates.

 

[James Suttles]

Not to Hijack the thread, I use the exact same workflow of your georeferenced aerial, I do exactly like you do, using a georeferenced aerial, and the deeds to create the search points. Once I find two of the points, that fit, I shift (translate) and rotate to those, and from there I am on the deed. When I use localization, is when for instance, I have a set of plans to stake a construction site, and the control is say something other than NAD83/2011. I set a base point in a wide open spot and using the RTN, which is at NAD83/2011, collect a point or you can use some known coordinate if its an open area. I then locate parts of the plan or boundary listed, do your localization from your SPC system, to the plan. An example is if you are dealing with something in NAD27 or NAD83/86.

I am staking a construction site right now, that the previous surveyor tied to NAD83/86, and he published coords on the plans, the control points listed with the coords were destroyed when I arrived on site, so I had to localize NAD83/2011 back to NAD83/86, in order to stake the site. This was also done to be sure that the coordinates in the ACAD file used for staking were relative to the localization. This could just have easily been a 1000.5000 coordinate system published on a set of plans from a Engineering or Architect group. I think we all know those cad files still exist, even in 2021.

 

[Matthew D. Sibole]

My workflow is the same as Mr. Suttles. Most of the time the surveys that I am following are not good enough to do a good localization on anyway. Just translate and rotate most of the time.

 

[Jim Frame]

3. If you need just to find points you can use alternative method, that is not creating new system, but moving base instead, to match surveyed point to target coordinates. It is M-Local.

What effect does M-Local have on exported vectors? And does it work with RTN, "moving" the transmitted RTN base coordinate? And is the effect of M-Local restricted to the page in which it's invoked, or does it affect coordinates on all pages?

 

[Michael Stazhkov]

What effect does M-Local have on exported vectors? And does it work with RTN, "moving" the transmitted RTN base coordinate? And is the effect of M-Local restricted to the page in which it's invoked, or does it affect coordinates on all pages?

In the field M-Local will shift the receiver real-time GNSS position to some vector independent of how the GNSS position was calculated and independent of a page. Previously saved points in the project with the same base coordinate also should be shifted.
The original coordinates calculated before applying the real-time shift also will be saved. So you can undo it.

 

[Jim Frame]

I think I like localizing better, as it limits the movement to the points on the local page. Having to remember to undo something sounds like a recipe for trouble to me.

 

[Shawn Billings]

Localization is the perfect tool for this application. M-Local is not the right solution.

Differences Between Localization and M-Local
Localization:
Localization creates the transformation parameters between two different coordinate systems. In Jim's example, he has coordinates that are like State Plane but are not actually State Plane. He doesn't know exactly what the difference is between his SPC-like local coordinates and true SPCS. These local coordinates have values that are similar to SPCS but are estimated from orthophotos and will be different from true SPCS in translation and rotation (and possibly scale as well). Localization will automatically determine these four parameters (four parameters for a 2D transformation: translate N, E, rotation around the Up-axis, scale, vs. seven parameters for 3D: translate N, E, U, rotation around N-axis, E-axis, Up-axis, scale).

The geodetic coordinates of survey points are not actually affected by localization. The base position and the vectors from the base to the rover determine the geodetic coordinates of all surveyed points. The localization only affects the geodetic position of design coordinates based on the transformation parameters determined by the localization. As the localization is altered, the geodetic coordinates of design points in the local coordinate system will change. Also when viewing surveyed coordinates expressed in the local system, the local coordinates of surveyed points will change if the localization is altered. So if Jim starts his localization with two points (the minimum required for translation and rotation), and stores this localization, he will have geodetic coordinates for all design points in that local system and he can view his surveyed points in the local system. He can then use this localization to reduce the size of his search area for the third point. At the beginning his search area is based on his ability to estimate the location of his geometry on an aerial photo, but after the first two points are localized (provided he has no blunders in his survey and design data and the points are spaced out reasonably well) he will have a much smaller target area to the third point. Supposing he recovers the third point, he can add this third point to the localization further refining the transformation. The result of this new localization using three points instead of two, however is that 1) the geographic position of design points will change but the grid coordinates will not 2) the local grid coordinates of survey points will change but the geodetic positions will not. This process can be repeated ad infinitum further refining the localization.

M-Local:
M-Local determines the difference in geodetic position between the base and the rover points collected from that base and a surveyed point with known coordinates. M-Local determines a simple 3 parameter transformation (translation in N, E, U) and applies this translation to the base and all points surveyed from that base. This is an excellent tool for making minor shifts (only a few meters) from a base that was started using autonomous coordinates to WGS84 or NAD83 coordinates, or for merging several base points with DPOS derived coordinates to one single coordinate value (likely only a few centimeters). Unlike localization, M-Local does change the geodetic position of surveyed points. M-Local does not affect the geodetic position nor the grid position of design points. In Jim's example, there is likely some rotation applicable in the transformation because he's taking geometry with an unknown relationship to North and estimating the rotation to SPCS using the aerial imagery. M-Local does not consider rotation (nor should it for the intended purpose of the application).

Localization vs. CoGo Rotate and Translate
In my opinion localization is superior to rotate and translate found in CoGo because localization uses all points in the localization to determine the rotation and translation, while CoGo rotate and scale uses one single point for translation and one single baseline for rotation. The likelihood is that all points in the two coordinate systems (local and geodetic) have some error, the notion of a single local point (design) and a single geodetic point (surveyed) having zero error is unlikely. Having said that, a good localization requires that the design geometry be relatively accurate. This will likely be from modern surveys using theodolite/EDM or total station, and from quality work done with transit and chain. Compass and chain surveys are not good candidates for multi-point localization, but are better suited for single point, single baseline rotations, like CoGo translate and rotate. It is possible, and in my opinion more convenient to do this in localization than CoGo by changing the single pair of points used for translation, setting the rotation using a second pair of points, and then locking the rotation after it is solved in the parameters screen and switching the second pair of points to "check". This forces the localization to only use the single design point/ survey point pair for translation and the second design/survey pair to contribute only to the rotation. This can be changed and saved again and again using different points as necessary to keep the localization related to the nearest found points.

Summary
Jim's localization appears to have worked properly in his last attempt. If the inverse between a particular design point and surveyed point showed the same distance that the residuals in the localization showed for that particular pair, then the localization was successful. I cannot explain the stakeout issue. From what I saw of Jim's localization, my one question would be pertaining to scale. For the purposes of developing search coordinates, it doesn't matter, but to be the most technically correct in my opinion, Jim would force the localization to use the most appropriate geodetic scale factor rather than allowing the software to determine the scale factor from the localization. The localization scale factor will be determined by a comparison of design and surveyed points, which should hopefully be very close to the actual geodetic scale factor. In this example Jim likely knows or can make an initial assumption that the design data is from an on-the-ground survey in which case, the scale factor should be the reciprocal of the combined factor. He can use the localization determined scale factor to help verify that assumption, but ultimately, before calling the localization finished he would decide how his design data linear units relate to his survey linear units and force the scale factor to that.

 

[Vladimir Prasolov]

Hi Jim,

Finally we found the root cause of the stakeout issue. The SEARCH points were imported as SPCS design points. The design points should be always imported in corresponding coordinate system due to fact they keep those coordinates internally forever. So localization does not affect those points at all because they already have SPCS coordinates.

The discrepancy could be seen by comparing design coordinates in point viewer and localization project.

For preventing similar problems in the future, I would suggest show original coordinate system for design points and original coordinates too in points view.

I re-imported points in localized system and attached the file. I hope it will work.
1280-108-210329 FIXED-210331 12_19_06.zip

Best regards,
Vladimir

 

[James Suttles]

Very good info, and I agree localization works good, if you are following a good verified closed boundary survey, but when you are following behind work that is suspect to its accuracy, I find its better to use rotate and translate, between what fits best. Shawn and Matthew both nailed what works.

Key note, is whatever error is in the work you are following, you will incorporate that same error, when you localize to that work.

It took me quite a few times to get localization down, and if I do not one ever so often, I will mess up and import the points to the wrong system. Importing to the correct system is the key, I think.

Thanks

 

[Vladimir Prasolov]

Very good info, and I agree localization works good, if you are following a good verified closed boundary survey, but when you are following behind work that is suspect to its accuracy, I find its better to use rotate and translate, between what fits best. Shawn and Matthew both nailed what works.

Key note, is whatever error is in the work you are following, you will incorporate that same error, when you localize to that work.

It took me quite a few times to get localization down, and if I do not one ever so often, I will mess up and import the points to the wrong system. Importing to the correct system is the key, I think.

Thanks

Hi James, We will reduce the probability of wrong importing system by adding original coordinate system and original coordinates in design point info. I hope it will help to recognize the issue earlier.

 

[Matt Johnson]

Hi James, We will reduce the probability of wrong importing system by adding original coordinate system and original coordinates in design point info. I hope it will help to recognize the issue earlier.

I thought this was shown at one point but I no longer see it. I posted many years ago that is very important to display the underlying coordinate system for design points, otherwise problems like this can occur.

 

[Vladimir Prasolov]

I thought this was shown at one point but I no longer see it. I posted many years ago that is very important to display the underlying coordinate system for design points, otherwise problems like this can occur.

Thanks Matt. Andrew implemented VSAPP-3354 that time. There is the check-box in Additional Filter:

But I think it is not enough due to nobody knows how useful that checkbox is when importing design points. And it is too deep in UI. So we decided to force that to be visible all time.

 

[Jim Frame]

Finally we found the root cause of the stakeout issue. The SEARCH points were imported as SPCS design points.

I'm pretty sure I imported them to a custom coordinate system. Doesn't this image indicate that?

But I'm wondering about the WGS84 indicator on the Surveyed side of the page -- does that reflect the SEARCH page underlying coordinate system, or the Surveyed points coordinate system?

The SEARCH coordinates as imported will be very close (within a foot or so) to SPC values, because I picked them off of a georeferenced image.

 

[Matt Johnson]

I'm pretty sure I imported them to a custom coordinate system. Doesn't this image indicate that?

But I'm wondering about the WGS84 indicator on the Surveyed side of the page -- does that reflect the SEARCH page underlying coordinate system, or the Surveyed points coordinate system?

No, WSG84 in that screenshot only indicates the underlying coordinate system of the localization. Pages do not have underlying coordinate systems, a single page can contain design points that have multiple different native coordinate systems.

 

[Jim Frame]

I'm giving it another try. I deleted all the design points, then set the SEARCH page to a new local coordinate system with SPC underlying. I then re-imported the design points, and re-did the localization, this time including today's shots, holding 3D for all 17 of them. That gave a good fit, with pretty small residuals, though with more tilt that I expected -- about 1° in one direction, just a few seconds in the other. Tomorrow I'll be re-tying all the points for redundancy, and I'll try staking out some to see if I finally got it right.

 

[Vladimir Prasolov]

I'm giving it another try. I deleted all the design points, then set the SEARCH page to a new local coordinate system with SPC underlying. I then re-imported the design points, and re-did the localization, this time including today's shots, holding 3D for all 17 of them. That gave a good fit, with pretty small residuals, though with more tilt that I expected -- about 1° in one direction, just a few seconds in the other. Tomorrow I'll be re-tying all the points for redundancy, and I'll try staking out some to see if I finally got it right.

Best luck Jim! We are always happy to help you! I think this discussion was very fruitful for all of us to clarify workflow and UI!

 

[Vladimir Prasolov]

I'm giving it another try. I deleted all the design points, then set the SEARCH page to a new local coordinate system with SPC underlying. I then re-imported the design points, and re-did the localization, this time including today's shots, holding 3D for all 17 of them. That gave a good fit, with pretty small residuals, though with more tilt that I expected -- about 1° in one direction, just a few seconds in the other. Tomorrow I'll be re-tying all the points for redundancy, and I'll try staking out some to see if I finally got it right.

Hi Jim. I would suggest to fix tilts to zero when the control points cover small area. It could be dangerous to extrapolate the tilt to bigger area. The heights matching is already improved by using geoid in underlying system.
Regarding pages, we created them for:
1. Grouping points and shapes to organize work.
2. Provide different representation attributes for displaying: coordinate systems and colors.

 

[Shawn Billings]

more tilt that I expected -- about 1° in one direction, just a few seconds in the other.

Jim, when running localization, I strongly recommend using an iterative approach (I know you use least squares adjustments -- think "minimally constrained"). Start off providing localization with the minimum amount of information required - two horizontal points and one vertical point. Of course the one vertical point can also be one of the two vertical points, but does not necessarily need to be one of the two horizontal points. Force the scale factor to the proper geodetic factor. If you suspect you are relating ground distances to an underlying SPC system, use the grid to ground factor. If your underlying system is NAD83 or WGS84, then use a factor of 1 (0 ppm). This is because the actual underlying system will be an oblique stereographic projection that is created at the average elevation of your surveyed points. The Helmert transformation used by localization cannot use geographic (i.e. latitude and longitude) coordinates, so it creates an intermediate projection to calculate the transformation).

First note the residuals of your two selected points with the scale factor forced to the proper geodetic factor. Both points will have identical residuals since the application places the difference evenly between the two points. Without the geodetic scale factor being used the residuals will be zero because the design system will be scaled perfectly to match the surveyed baseline. Second, look at the rotation. If your design geometry is fairly reasonably close to a North orientation, you should be able to look at the rotation and see that it matches your expectation. If you accidentally flipped your design and surveyed points, the rotation will be closer to 180 instead of being closer to 0.

With your initial localization done, you can then add more points. Use the "Check" to see if the points are within the expected tolerance. If so, then add the new point horizontally. You'll notice the residuals will go up as you add more points initially.

Vertically, I almost always turn off tilts by forcing the tilt values to zero. The only exception I can think of would be if I were working in an area with a poor geoid model. In such a case localization can be used to create a sort of simple geoid model (it will have only one slope so it would probably have limited application in such a situation). It is important however that if you are turning off the tilt that your underlying coordinate system have a geoid model!! You can then follow the same approach used for horizontal. Add any vertical only points as "check". Any points that are used as Horizontal only will already show the vertical as a "check" as well. Once you see these points show acceptable residuals, you can change them to vertical or 3D. With the tilts set to zero, the effect of vertical points being added to the localization is that the entire design is translated vertically by the average of the point pairs.

Using an iterative approach to localization is absolutely critical. As Vladimir stated, tilted localizations can be particularly dangerous when extending beyond the geometry of the localization (working outside the box). But turning tilts off allows the user to work well beyond the localization geometry. Using the proper geodetic factor for the horizontal solution is similarly important for the same reasons (also, I don't know anyone who would dial the ppm on their total station to match the previous survey, which is exactly what happens when using the scale factor generated from a localization).

Localizations are a very valuable tool. Excellent for getting related to an existing survey and excellent for evaluating the quality of the relationship between two coordinate systems. But it is a very technical application and should be used with care. If you haven't done so already, please watch the video I posted above. It discusses pitfalls like your 1 degree tilt and how to avoid it.

 

[Jim Frame]

I appreciate the cautions, but should note that -- to date, anyway -- my only use for localization is to search for existing points. For the specific job I'm playing with now I don't need it at all, I'm just using the job as an opportunity to get the procedure down pat.

The reason I was surprised by the 1° tilt is that this is a control job -- I was retained by a city to validate control established by another surveyor for a bridge construction project. The city wants to be sure that when the contractor starts building from both ends he'll meet himself in the middle. I don't have metadata yet, but I'm beginning to suspect that the first surveyor used RTK to position the points, rather than static and total station as I initially assumed. The fact that he set some flight control points right under trees is beginning to make me think I may have to do more than just check...

 

[Shawn Billings]

I appreciate the cautions, but should note that -- to date, anyway -- my only use for localization is to search for existing points. For the specific job I'm playing with now I don't need it at all, I'm just using the job as an opportunity to get the procedure down pat.

The reason I was surprised by the 1° tilt is that this is a control job -- I was retained by a city to validate control established by another surveyor for a bridge construction project. The city wants to be sure that when the contractor starts building from both ends he'll meet himself in the middle. I don't have metadata yet, but I'm beginning to suspect that the first surveyor used RTK to position the points, rather than static and total station as I initially assumed. The fact that he set some flight control points right under trees is beginning to make me think I may have to do more than just check...

I know you are a consummate professional, so I have no doubt about how you are going about using localization and what you are doing is exactly the way to perfect your knowledge of it.

You should be able to identify the problem points very efficiently with the process I describe above.