Early in 1998, mapping operations for the Mars Observer Camera (MOC) will begin. Since no images of Mars exist at MOC resolution, and the MOC will not acquire global coverage at resolutions even approaching those available from previous missions, target identification must be done using the existing Mariner 9 and Viking image collections. The high-resolution system of the MOC has a pixel resolution of 1.4 meters/pixel and a field of view of only 3 km. It would be desirable to use digital maps of at least 50 meters/pixel resolution to choose MOC targets. Unfortunately, the best available digital map of Mars, the USGS Mars Mosaicked Digital Image Model (MDIM), has a resolution of only 230 meters/pixel. Direct construction of higher-resolution maps from spacecraft images is problematic because of poor knowledge of the geometry in which most of the images were acquired. To have a 50% chance of acquiring a MOC image of a particular target, knowledge of target locations to accuracies on the order of half a MOC field of view, or 1.5 km, are required.
This paper describes the status of our effort to address these problems as of the end of 1994. We explore the history of the existing datasets, describe a method for using the MDIM as an implicit control network to which images can be registered, and report the results of applying this method to a large subset of the Viking images, using both manual and automated techniques. Finally, we close with a discussion of the work left to be done.
The existing datasets consist of the raw imagery, geometric or "SPICE" information for that imagery, two control point networks, and the MDIM.
A spacecraft image is accompanied by ancillary information about the circumstances under which it was taken. This has been termed a "SPICE kernel", SPICE being an acronym for Spacecraft, Planet, Instrument, Camera, and Event. For the purposes of registering an image to the MDIM, we are concerned about the geometric information: the orientation of the camera (the "C matrix"), the location of the spacecraft relative to the planet (the "S vector") and the orientation of the planet (the "P matrix"). Each of these quantities can be represented as a triple of real numbers: the vector in Cartesian coordinates, and the orientation matrices as Euler angles.
Typically, it is assumed that the S vector (derived by radiometric tracking of the spacecraft and the ephemeris of the planet it orbits) and the P matrix (defined by convention, based on reduction of previous observations of planetary features) are known to fairly high precision. For Mariner 9 and Viking, the C matrix is a composition of the spacecraft orientation in an inertial frame, derived from star sensor information, and camera orientation in the spacecraft frame, measured by encoders on the axes of the scan platform. The resulting C matrix often has large errors, and this constitutes the major barrier to registration of the image to the planetary reference frame.
Assessing the quality of the SPICE data is difficult. The earliest version of the Viking SPICE data available at MSSS is that delivered with the PICS image processing system from USGS. (Although we believe that this dataset was derived from the original JPL SEDR dataset, there appear to be some discrepancies between the two; we have been unable to obtain the original SEDR to examine the differences.) We also have 5,293 updated SPICE kernels computed as a side-effect of the MDIM production process; these are available on the CD-ROM volume USA_NASA_PDS_VO_2007. The quality of the original dataset, for these images, can be gauged by computing where a fixed point on the camera's focal plane intersects the planet for each image, and examining the differences between the results using the original and the MDIM-produced SPICE data. This reveals some errors of over 60 degrees in latitude and 150 degrees in longitude, but 98% of the images have errors of less than 5 degrees. The errors are plotted below.
Knowledge of the exact locations of features on a planet's surface is part of the province of geodesy. Typically, the knowledge is given in the form of a control point network, a list of identifiable features and their coordinates in some reference system. The history of the effort to define a control point network for Mars is somewhat confusing. [Davies92] provides a simplified account; the ongoing reports of the groups involved (e.g., [Davies84], [Davies86], [Davies91], [Wu86], [Wu90]) give a more detailed picture. The original effort, led by Merton Davies of the RAND Corporation, began in 1969 with Mariner 6 and 7, and continued on through Mariner 9 and Viking. The RAND network has been defined purely by traditional photogrammetric techniques, and was initially intended for horizontal control. A second control network effort, by Sherman Wu and others at the U.S. Geological Survey (USGS), began in the early 1980s with Viking data and was originally intended for vertical control. It used both photogrammetric methods and other information sources, such as pressure measurements derived from Mariner 9 ultraviolet spectrometer data, spacecraft occultation data, and range measurements made by Earth-based radar. It was tied to the RAND horizontal network as it existed in 1978, and apparently then evolved independently as it was used to support the more general mosaicking activity at USGS. Both the USGS and RAND networks have been extended and refined, more or less independently, since that time.
At present, both control nets have a stated error of about 5 km globally. The RAND net, being based on traditional photogrammetric strips, has higher accuracy (about 3 km) along the strips, and is extremely accurate (on the order of 50 meters) in the vicinity of the Viking landing sites. The USGS net is sparser, and has a standard error of about 5 km. Versions of both control nets have been received by MSSS in machine-readable form; the RAND net [Colvin92] has 9584 points and the USGS net [Duck91] has 4506 points.
It is implied in [Davies92] that about 2,200 points in the USGS control net correspond to points in the RAND net. By comparing the coordinates of the points in each net, we found 2,871 points that appear to correspond (the point identifiers of the USGS points are those of the RAND points plus 20,000.) We have discovered two discrepancies. First, for ease of measurement some of the USGS points have been moved from the centers of craters to their rims [Wu92]. Second, the defined rotational pole and central meridian of Mars have been updated for the RAND network but not for the USGS network. These changes result in differences between corresponding points in the latest versions of each network on the order of 12 km in longitude; these differences are plotted below.
Our original intent was to use one or the other of these control networks, suitably transformed, to compute pointing information for Viking images directly. However, it quickly became obvious that neither net was dense enough for this purpose; at best, a typical Viking image of the desired resolution might contain one or two control points.
Another difficulty in using the control networks directly is uncertainty about what actual feature a control point is associated with. For the RAND control net, we have a listing of the image coordinates of each control net point; from this, we could build a catalog of image features and their corresponding control points. No similar information has been obtained for the USGS control net, although for that net, a direct comparison with the MDIM is possible.
Because of the difficulties with direct usage of control point coordinates discussed above, we decided instead to adopt the USGS MDIM as a "virtual" control network. Since the MDIM images are map projections, the latitude and longitude of any feature can be computed directly from its pixel coordinates. The radius can be obtained either by assuming all points lie on the USGS spheroid, or by extracting the altitude from the USGS DTM. In this manner, any feature that whose pixel coordinates can be measured in both a Viking image and the MDIM can be used as a control point to register that image to the MDIM.
The use of the MDIM in this manner raises the question of the MDIM's accuracy. While the USGS control net served as a basis for control of the MDIM, it is stated in the MDIM documentation [Eliason91] that
Discrepancies between adjacent frames are far less than 20 pixels over most, but not all, of the planet. We attempted to distribute the error so that it was not obtrusive in the mosaics, but this was not possible in some areas. ... Given assumed camera positions, camera orientations were derived by reducing to their minima the discrepancies between images in overlapping frames and the control net.
This implies that errors in some parts of the MDIM are significantly higher than the 5 km error of the USGS control net. We know of no quantitative way to characterize these errors using only the data in hand. However, an admittedly non-exhaustive search for large discontinuities along the seams between the individual images of the MDIM shows only a small number of problems:
lat lon problem -22.12 48.84 2.5 km mismatch 39.65 304.35 mismatch 26.85 31.32 bisected crater -25.8 177.4 incorrect image in mosaic 13.87 153.94 bisected crater -13.76 348.06 bisected crater
Regardless of its actual accuracy, then, the MDIM appears to be suitable as a fairly smooth and self-consistent reference frame. Once points are defined relative to the MDIM frame, they may be transformed to a more accurate reference frame by a variety of techniques. For example, the difference in rotational pole between the USGS and RAND control nets can be modeled as a simple rotation; more local adjustments can be modeled as translations and rotations or general warps in a suitable coordinate plane.
For a single image, given the S vector and the C and P matrices, we can relate a feature at a measured location (Ix, Iy) on the camera's focal plane to the feature's location in 3-space (Lx, Ly, Lz) as follows:
S' = CPT (S - L)
Ix = S'x / f
Iy = S'y / f
However, if the C matrix has significant error, then the resulting focal plane location will be in error. But if we know P, S, and L for three or more points on the focal plane, then we can compute C directly with no a priori knowledge. (Three points determine a value for C; more points overdetermine the value of C but can be used in a least-squares sense. To reduce the effects of misidentified points, we require each image to have at least four measured points, discarding the point with the highest error from the final fit.)
This can be stated mathematically as:
CA = B
where C is the 3x3 C matrix, A is a 3xN matrix whose rows are the individual unit vectors to the control points in camera coordinates, and B is a 3xN matrix whose rows are the unit vectors to the control points in inertial coordinates. Each of the latter is simply {P T} ( Si - Li ). The C matrix can be solved for in a least-squares sense using standard methods such as Singular Value Decomposition [Press88]. Unfortunately, the resulting matrix is not guaranteed to represent a pure rotation. Since it should be close to a pure rotation, however, we orthogonalize it using a Taylor series expansion.
Using this algorithm, we can register a Viking image to the MDIM by locating four or more common features on the image and the MDIM and specifying their pixel coordinates. Once done, a new C matrix can be computed for the image which minimizes the mismatch between feature locations; the residual error can be used as a metric for the quality of the registration.
Having developed a plan of attack, we then proceeded to register individual Viking images to the MDIM. To minimize data storage requirements, we restricted the set of images to those with resolutions better than 400 meters/pixel and an emission angle of 45 degrees or less. We transferred the copy of the Viking dataset on 6250 BPI 9-track tape at MSSS to 2GB 8mm Exabyte cartridges. 35,325 of the resulting 49,257 images (omitting calibration and encounter images) were identified from geometric information as being candidates for targeting. We copied these images from 8mm cartridge to a 60GB magneto-optical jukebox, succeeding in transferring 33,266 images. (The missing images resulted from various problems, such as bad cataloging information, bad 6250 BPI tapes, and errors during the transfers to 8mm or from 8mm to the jukebox.)
The remaining images were then divided into 3 subsets: low resolution (100 to 500 m/pixel), medium resolution (50 to 100 m/pixel), and high resolution (50 m/pixel or higher). The subset of low resolution images were passed to an automatic control point extraction and registration program. If the automatic processing was unable to compute a corrected C-matrix within acceptable error limits, the image was registered manually. All other images were registered manually.
Based on geometric and cataloging information, 34,624 images were candidates for registration. The breakdown by resolution was:
50 m/pixel or better 14,865 50-100 m/pixel 8,467 100-400 m/pixel 11,292 total 34,624
Registration was attempted in two ways: using an automated point-matching algorithm (described more fully in an appendix), and manually. It was hoped that the automated procedure would be effective enough to register the majority of the images, but this proved not to be the case. 10,359 images were examined automatically. Of these, 2,503 were rejected before point matching because of poor image quality or because insufficient geometric or cataloging information existed for them. 3,546 images had enough control points measured to allow computation of the updated C matrix.
13,902 images were examined manually. Of these, 10,591 had enough control points measured for C matrix updating; 137 additional images failed to be registered after control point identification.
Essentially all of the processable images from 400 m/pixel to 50 m/pixel have been included in the current dataset. Those missing fall into one of the following categories. (When possible, estimates of the number of images in each category are given in parentheses.)
We found images at resolutions better than 50 meters/pixel difficult to register to the MDIM because of the difference in scale. We had planned to "bootstrap" to higher resolutions by registering images at somewhat higher than MDIM resolution, then registering the higher-resolution images to maps constructed using the updated C matrix information. Unfortunately, due to software problems the activity was terminated before this could be done on more than an experimental basis. However, it did prove possible to register some images of roughly 50 meter resolution to the MDIM directly, and 404 images of this type are included in the dataset.
As a metric of the registration's quality, we computed the average arc distance between control point locations and their locations when transformed from the image plane to planet using the new C matrix. Using this metric, the average error of the entire dataset is 0.62 km. 13,714 images have an error of 1.5 km or less.
Two datasets have been produced. Short samples of each dataset are shown here; the full datasets are available in electronic form by contacting the authors.
The first, unprocessed dataset consists of the measured locations of features on the Viking images, the corresponding locations of the features on the MDIM in latitude, longitude, and radius, and a 20-term bicubic function describing the vidicon distortion in the image. The latter has been derived by measuring the locations of the reseau marks in each image and comparing them to the nominal reseau locations.
The reseau-fitting terms p0 through p9 and q0 through q9 are evaluated using the expressions
x corrected = x + p0 + p1 x + p2 y + p3 xy + p4 x2 + p5 y2 + p6 (xy)2 + p7 x3 + p8 y3 + p9 (xy)3
ycorrected = y + q0 + q1 y + q2 x + q3 xy + q4 y2 + q5 x2 + q6 (xy)2 + q7 y3 + q8 x3 + q9 (xy)3
The control points are given in two separate files, one for the manually-defined points and one for the automatically-defined. Note that in many instances, the automatic procedure tended to locate points that would not have been selected by a human; this will probably reduce the utility of these points for future applications.
The latitude is given in geodetic form based on the USGS spheroid, with an equatorial radius of 3393.4 km and a polar radius of 3375.8 (a flattening of 5.1865 10-3.)
The image coordinates are in units of integer pixels, referenced to (0,0), the upper left corner of the image. Before use in the registration algorithm, these coordinates must be corrected for vidicon distortion; we removed this distortion using the function given earlier.
picno X Y latitude longitude radius (km) (geodetic) ___________________________________________________________________ 003A01 320 548 20.384766 37.218712 3389.236881 003A01 832 674 20.060547 37.568616 3389.306851 003A01 878 428 19.904297 37.418989 3389.297374 003A01 932 828 20.056641 37.726635 3389.307617 003A01 752 992 20.267578 37.747762 3389.254101
The second, more generally useful dataset is derived from the first, and contains the updated C matrix information for each image. The dataset is provided in two files, one for manual processing and one for automatic. Following the conventions of the Viking Project, the C matrix is given in Euler-angle form in the B1950 coordinate system.
picno declination RA twist __________________________________________________ 003A01 5.602768 229.722845 266.610960 003A02 4.732229 228.931367 267.206609 003A03 6.535181 229.018903 266.904891 003A05 7.419383 228.548791 267.820553 003A06 6.492269 228.048256 268.321937 003A07 8.295956 228.074604 268.872719
Roughly 3,500 images in the dataset correspond to images for which updated SPICE information exists from the USGS MDIM production process. This information can be used as an independent test of the quality of the MOC dataset. The following plots show the difference between the USGS and the MSSS SPICE for 1,699 automatically-registered and 1,952 manually-registered images. Each point is the average difference of the planetary coordinates of 16 principal points on the image plane, obtained using the USGS and MSSS C matrices.
For both plots, about 67% of the images differ less than 1.5 km in both directions. About 90% differ less than 3 km. We have examined some of the cases with the largest errors to determine which dataset is inaccurate. This was done by map-projecting the images using the USGS and MSSS SPICE values, and then manually comparing the locations of actual features on the MDIM with their locations as shown on the map projections. The following plots show the results for about 25 of the images with the largest differences:
Since this comparison is a manual process, making it for large numbers of images is impractical. These results suggest that the manual MSSS dataset is more significantly more accurate than the USGS dataset, but the situation is reversed for the automatic MSSS dataset. Examination of the worst cases for the latter show that the control point correspondences derived automatically are simply incorrect in those cases.
Without manually examining each case, it is difficult to know how much the automatic dataset should be trusted. The statistical difference between the USGS dataset and the MSSS dataset is largely the same for the manual and automatic subsets, suggesting that the automatic processing performed well most of the time, but had a larger number of failures than did manual processing. Unfortunately, the quality metric of residual control point error does not provide a measure of confidence for the automatic processing, making it difficult to distinguish a good case from a bad one. Therefore, we suggest that the automatic dataset be applied cautiously.
The MSSS dataset makes it possible to know, with high accuracy, where an image is on the surface of Mars relative to the MDIM reference frame. This should make it possible to construct new mosaics without manual intervention, and software to do this is under development. In many areas, such mosaics could have quality significantly higher than that of the MDIM.
Unfortunately, the original goal of this effort, targeting for the MOC, remains largely unaddressed. Fewer than 5% of the Viking images of useful resolution have been registered to the MDIM. Even if it were desirable, budgetary constraints make a resumption of the manual registration effort unlikely. Future effort will focus on the development of more reliable automatic algorithms for feature location and matching; some promising results have been had using a method that relies on explicit knowledge of the lighting direction to locate craters.
Jeff Warren developed the registration algorithm on which this work is based. Greg Crinklaw implemented the Viking image processing software used. Obviously, the lion's share of credit goes to the people who performed the tedious task of manual image registration with such care: Brian Boyle, Matthew Fishburn, Brett Palluconi, and Suzanne Rupert. Finally, we thank Mike Malin, the MOC Principal Investigator, whose patience and encouragement made this effort possible. The work described here was supported by the Jet Propulsion Laboratory, Contract 959060 to Malin Space Science Systems, as part of JPL Contract NAS-7-918 with the National Aeronautics and Space Administration.
[Colvin92] Colvin, Tim, personal communication, 10 November 1992.
[Davies92] Davies, M., Batson, R., and Wu, S.S., "Geodesy and Cartography", in Mars, edited by H. Keiffer et al, University of Arizona Press, 1992.
[Davies91] Davies, Merton E. and Rogers, Patricia G., "The control network of Mars: April 1991", Reports of Planetary Geology and Geophysics Program, 1990, p 497-498
[Davies86] Davies, Merton E., "The control network of Mars: October 1986", Reports of Planetary Geology and Geophysics Program, 1986 p 536
[Davies84] Davies, M. E., "The control network of Mars: October 1984", In NASA, Washington NASA Reports of Planetary Geol. and Geophys. Program, 1984, p 558
[Duck91] Duck, Bonnie, personal communication, 18 September 1991.
[Eliason91] Eric Eliason, Raymond Batson, and Anthony Manley, documentation file on CD-ROM volume MG_2001, "Mars Mosaicked Digital Image Model (MDIM) and Digital Terrain Model (DTM)", United States Geological Survey, August 1, 1991 (Version 1.0)
[Gaskell88] Gaskell, R., "Digital Identification of Cartographic Control Points", Photogrammetric Engineering and Remote Sensing 54, 6, June 1988, pp. 723-727.
[Press88] Press, William
[Wheeler93] Wheeler, K., "A computer vision system for tracking
proliferating cells", M.Sc. thesis, Centre for Intelligent Machines,
McGill University, August 1993.
[Wu86] Wu, Sherman S. C. et al, "Progress in compilation of the
1:2,000,000-scale topographic map series of Mars", NASA Reports of
Planetary Geology and Geophysics Program, 1986 p 530
[Wu90] Wu, Sherman S. C. et al, "Publication of topographic atlas and
control network of Mars", NASA Reports of Planetary Geology
and Geophysics Program, 1990 p 499-502
[Wu92] Wu, Sherman, personal communication, 12 April 1992.
The geodesy effort for the MOC began in 1990, when the initial version
of the registration algorithm was developed by Jeff Warren. In 1992,
Kim Wheeler began work as the MOC Navigation Engineer, developing the
automatic feature location and point-matching algorithms, writing an
interactive program for manual registration, and extending the
registration algorithm. The Viking images were transferred to the
jukebox in March 1993, and the automated registration activity was
performed from April to August. Manual processing began in mid-1993
as a pilot activity; following the loss of Mars Observer in August,
three members of the MOC operations staff began a major effort of
manual registration, which continued until January 1994 when both
Wheeler and the remaining operations staff left MSSS.
The goal of point matching is to locate features visible on both the
MDIM and on a geometrically-raw Viking frame and create a list of
correspondences. Humans are fairly good at this task, especially if
the images are of similar scale and orientation, but exactly how they
perform it is still a subject of considerable debate and research.
Vision processing algorithms implemented on computers often rely on
"brute-force" approaches. For this work, we used algorithms
originally developed for tracking of biological objects in microscope
fields [Wheeler92]. The general processing flow for each image is:
The above processing is applied to both Viking frames and the MDIM.
The result is a collection of raw points whose correspondences with
each other (if any) are unknown. Correspondences are searched for
using a iterative algorithm that starts with an initial guess based on
existing SPICE information. The guess is updated assuming that the
actual registration differs from that guessed by only a rotation and
translation. A new C matrix is derived at each step, and the errors
of each point correspondence are examined; correspondences with large
errors are discarded and replaced with other candidates if available.
Candidates are chosen from annular regions surrounding each point.
Processing stops when enough correspondences with errors less than a
fixed value have been located to derive a C matrix.
It should be noted that this algorithm does mot make use of any
similarity between the features themselves, such as was used in
[Gaskell88]. While naive, this property makes the algorithm
insensitive to the variations in lighting and image quality present in
the MDIM. While the algorithm may identify many fewer points than
would a human, only a half dozen or so points are required to update
the C matrix.
Even so, the algorithm's success rate is not high. Only about a third
of the low-resolution images could be registered automatically, and
that fraction fell off to nearly zero at medium resolution. More
robust algorithms will be the subject of future research.
To gain some insight into the issues surrounding the use of SPICE
information for Viking, consider the randomly-chosen Viking image
524A30. The acquisition time of this image is given as JD
2443471.154101 in the PICS SPICE. Assuming this is a Julian Ephemeris
Date, it corresponds to 11/23/1977 15:41:06.143 as computed by the
NAIF toolkit. The acquisition time is given as 11/23/1977 15:41:06.31
in the MDIM geometry files; the source of the 167 millisecond
discrepancy is unknown. The JPL 6250 BPI file header gives an
"OET-GMT" of 11/23/1977 15:41:06, as does the header of the raw Viking
image on CD-ROM VO_1020. (We originally made the erroneous assumption
that the "JD" was a straight Julian Date, which corresponds to a time
of 15:41:54.326 due to the addition of leap seconds.)
The original PICS SPICE values were:
C matrix = (3.228072881698608, 164.6398010253906, 245.4609069824219)
S vector = (9031.347656250000, -2628.588867187500, -656.3513183593750)
P matrix = (52.69398498535156, 317.3118591308594, 328.5455932617188)
As reported on the USGS MDIM disk, the values were:
C matrix = (2.97846, 164.52330, 245.62589)
S vector = (9031, -2629, -656)
P matrix = (52.694, 317.312, 328.546)
Note that the components of the S vector have been rounded to integer
kilometers.
For processing at MSSS, we have made two assumptions. First, we
assume that the coordinate system used, EME1950, is identical to the
B1950 coordinate system supported by the NAIF toolkit. We also assume
that the planetary orientation used by the USGS throughout the
construction of the MDIM is the same as the one reported in the
geometry files, not the one currently defined by the International
Astronomical Union. There are small discrepancies between the original
P matrix, the one reported by USGS, and the IAU definition as computed
by the NAIF toolkit:
P orig = (52.69398498535156, 317.3118591308594, 328.5455932617188)
P MDIM = (52.694, 317.312, 328.546)
P IAU = (52.694193, 317.312080, 328.759038)
The mismatch in the third angle (the planetary rotation angle W)
corresponds to an error of over 12 kilometers. In the spirit of using
the MDIM as an absolute reference system, we have chosen to use the
older P matrices. However, this discrepancy must be reconciled before
MOC targeting commences, since the software written to support MOC
operations uses the IAU reference frame.
In the MSSS dataset,
C matrix = (2.989443 164.537379 245.528627)
Both C matrices register the image to the MDIM with residual errors of
about 1.5 km.
Appendix -- History of the processing effort
Appendix -- the automated point-matching algorithm
Appendix -- An example