User Manual 3.3 Ephemeris comparator
Introduction
Scope
This tool can compare two ephemeris files.
The Ephemeris Comparator Tool can compare the raw data: for each column, it computes the absolute and relative deviation between each element. It can also perform a stamped comparison : In this case, the user configures the columns as well as the type of the object meant to be compared, and the tool computes the meanginful deviation between each of these objects (for instance the euclidian distance between two vectors).
The tool also provides statistical data, such as max deviation, mean deviation and standard deviation for a column. It can also compare the deviations to user-given thresholds and return the gap between both (or, as a negative number, the amount by which the deviation overcomes the thresholds).
Javadoc
The ephemeris comparator is available in the package fr.cnes.sirius.patrius.tools.ephemerisComparator of the PATRIUS library.
| Library | Javadoc |
|---|---|
| Commons Math addons | Package fr.cnes.sirius.patrius.tools.ephemerisComparator |
Links
None as of now.
Useful Documents
None as of now.
Package Overview
Features Description
Ephemeris comparator tool
The Ephemeris Comparator tool provides the following services :
- Get streams from both data source files,
- Get the information embedded in these streams,
- If need be, build objects out of some of the data,
- Compute the absolute or relative deviations between the columns,
- Compute statistical data on these deviations, such as mean, maximum and standard deviation,
- Compute the margins between the deviations and user-provided thresholds,
- Produce a report (the result is stored in 3 distincts files : synthesis, absolute results and relative results).
Contents
Interfaces
| Interface | Summary | Javadoc |
|---|---|---|
| DeviationProvider | This interface specify, the method for the computation of a deviation.. | ... |
Classes
| Class | Summary | Javadoc |
|---|---|---|
| AbstractDataLoader | This abstract class is the base for a DataLoader. | ... |
| ComparisonData | This class represents a set of results from a comparison of data objects | ... |
| Data<T> | This class represents a set of columns, which values are Ts. | ... |
| DataComparator<T> | Parent to the data comparators, this class provides the main computeComparison function. | ... |
| DateDPFacade | This class is a facade for the use of dates. | ... |
| DeviationProviderFactory | This is a factory to simplify the construction of DeviationProvider objects. | ... |
| DoubleDPFacade | This class is a facade for the use of double precision real numbers. | ... |
| EphemerisComparator | This class provides the global usage of the ephemeris comparator library. | ... |
| EphemerisComparatorMain | -main of ephemeris comparator | ... |
| MeaningfulData | This class is a container for meaningful data. | ... |
| MeaningfulDataComparator | This class compares meaningful data in DeviationProvider form. | ... |
| PSimuDataLoader | This Loader is designed to extract the data from a PSimu formatted ephemeris file. | ... |
| PVCoordinatesDPFacade | This class is a facade for the use of PVCoordinates. | ... |
| QuaternionDPFacade | This class is a facade for the use of quaternions. | ... |
| RawDataComparator | this class is used to compare two sets of data, possibly using thresholds. | ... |
| Report | Production of a report and log files. | ... |
| RotationDPFacade | This class is a facade for the use of dates. | ... |
| Setup | The purpose of this class is to load and represent the setup data for the ephemeris comparator. | ... |
| ThresholdsData | This class contains the information on the margins of the deviations with regard to a set of thresholds. | ... |
| VectorDPFacade | This class is a facade for the use of vectors. | ... |
The Ephemeris Comparator
Conception of the Ephemeris Comparator
Get streams from both data source files
The different DataLoaders such as PSIMUDataLoader get streams thanks to the same method, described and implemented in the subclass, AbstractDataLoader. This class provides two getStream methods, one with a File parameter, and the other with a URL. This distinction allows the seperation of errors related to getting a Stream and using it.
Get the information embedded in these streams
The getData method is specific to each DataLoader. Indeed, every ephemeris file has a specific format, which means that parsing has to be adapted to the structure of the file. This is done by having one dedicated DataLoader per format type. For example, the PSimuDataLoader is dedicated to parsing an ephemeris file using the PSimu format.
The parsing methods use regular expressions extensively in order to recognize patterns (e.g. data columns). In future releases, this can be used to get more data from the ephemeris file (e.g. the name of each data column).
If need be, build objects out of some of the data
The stream reading provides a Data object, which is a simple container based on a list of columns of simple values : ArrayList<ArrayList<Double>>. Similarly, one can broaden the type of values contained in the columns, and obtain a MeaningfulData object that contains a list of columns of DeviationProvider, e.g. quaternions or vectors. This object can be built out of the Data object, by putting the columns indexes and the types they effectively represent in the setup file.
A new comparator, MeaningfulDataComparator, has also been added to compare such objects. It provides ComparisonData objects, and therefore do not change anything in the report generation.
Finally, in order to compare meaningful objects, new DeviationProvider classes have been implemented. These classes are facades to the usual ones, and allow us to add and control the computeDeviation method.
All these changes are shown in the graph of the Package Overview section (these classes works in parallel of DataComparator).
Compute the absolute or relative deviations between the columns
The DataComparator class provides methods to compute the absolute and relative deviations between two Data objects.
Note : For enhanced evolutivity, data computation is separated from data representation. As such, the comparison methods are seperated from the Data objects and therefore, comparison is done using the DataComparator object.
The deviations computation have been implemented directly in this class because the comparison provided by the base do not return the deviations.
Compute statistical data on these deviations, such as mean, maximum and standard deviation
The comparison returns a ComparisonData object, which extends the Data object. It is dedicated to holding deviations, and give the user the possiblity to compute the mean, maximum and standard deviation of each column. We used the StatUtils class of Commons Math product to compute data.
Compute the margins between the deviations and user-provided thresholds
ComparisonData also provides a method to comparison computed deviations to used-provided thresholds, thus producing a ThresholdData object (also an extension of Data). This class can provide information on the margins. It can give the number of acceptable deviations, the number of refused deviations, and the line number of the first refused deviation. This latter can help analyse data, e.g. by showing where the dates of each file no longer match.
Generate a report
The report consists of 3 files :
- Synthesis file : this file is an XML file content the following data
- date of generation
- absolute path of the used ressource file
- absolute path of the fist file to compare
- absolute path of the second file to compare
- by column :
- the name of the column
- the mean relative deviation
- the maximum relative deviation
- the mean absolute deviation
- the maximum absolute deviation
- the standard deviation
- the number of acceptable margins
- the number of refused margins
- the line number of the first margins
- Absolute results : this file is an ACSII file with <TAB> separator. The fist row contains the column headers. Subsequent rows contain the absolute gaps.
- Relative results : this file is an ASCII file with <TAB> separator. The fist row contains the column headers. Subsequent rows contain the relative gaps.
Configuring the Ephemeris Comparator
At the moment, both need a properties file to perform tasks. Below is an example of a properties file as well as a description of its contents :
COLUMN_NUMBER = 4 COLUMN_TITLES = Position Speed Angle QSW COLUMN_TITLES_LOF = x y z vx vy vz THRESHOLDS = 1.0e-6 1.0e-9 1.0e-7 LOF THRESHOLDS_LOF = 1.0e-8 1.0e-8 1.0e-8 1.0e-6 1.0e-6 1.0e-6 FILE1_TYPE = PSIMU FILE1_COLUMNS = VECTOR 2 3 4; VECTOR 5 6 7; 8; QSW 2 3 4 5 6 7 FILE2_TYPE = PSIMU FILE2_COLUMNS = VECTOR 4 3 2; VECTOR 7 6 5; 10; QSW 4 3 2 7 6 5
COLUMN_NUMBER This file encloses all the information needed for the setup of the comparator tool. The first line, COLUMN_NUMBER, tells the number of columns that the tool will compare. This is not the number of columns used in the ephemeris files, but rather the number of column compared. For instance, it could be useful to compare the vector column built out of the file columns 2, 3 and 4, and also compare each of these columns. In this case, we use 3 columns in the files, but compare 4 columns. Hence, in this case, COLUMN_NUMBER = 4.
COLUMN_TITLES,COLUMN_TITLES_LOF The titles of these columns as they will appear in the report are given in COLUMN_TITLES and COLUMN_TITLES_LOF for LOF objects (the LOF column titles should begin with "QSW" or "TNW" word).
THRESHOLDS,THRESHOLDS_LOF Similarly, the thresholds against which to compare the deviation between the two files are entered in the THRESHOLDS property and THRESHOLDS_LOF for the specific LOF objects. In this case, "LOF" must be noted in the property THRESHOLDS to the location of LOF object.
FILE1_TYPE,FILE2_TYPE The last section characterizes the files and what to take in them. The FILE1_TYPE and FILE2_TYPE properties are used to choose the correct parser to retrieve the information. At the moment, the PSIMU loader is the only one available. It can extract the information from files where there are simple columns of numbers. It can parse numbers with the same format as the Java Double parser (eg : 5, 6.3, -159.235411, 1.7562e-11 or +7.57E15). The available file types are listed in the AllowedFileType enum.
FILE1_COLUMNS,FILE2_COLUMNS These fields, concerning the data files, are used only with a meaningful comparison. The FILE1_COLUMNS and FILE2_COLUMNS properties tells which columns to compare and which column to build. The columns that will be compared and that will appear in the report are separated by semi-colons. A column description can be a simple number, and will then be interpreted as a Double column, meaning that it will use the comparison associated with the Double type. Then, if the user wants to explicitly give meaning to a column, that is to say build a column of objects instead of numbers, he only has to write the expected type followed by the number of columns to use in the source files. In the example, we build a vector from the column 2, 3 and 4 of the first source file. The possible types are listed in the DataType enum, and are the following, with number of accepted columns :
- Double (DOUBLE) : 1
- Vector (VECTOR) : any number
- Angle (ANGLE) : 1
- Date (DATE) : 2, 3, 4, 6, 7
- Julian day 1950 : elapsed days since the 01/01/1950 at 00:00:00, elapsed seconds in the current day
- year, month, day (the time is set to 00:00:00)
- year, month, day, elapsed time from this date
- year, month, day, hour, minute, second
- year, month, day, hour, minute, second, elapsed time from this date
- Rotation (ROTATION) : 3, 4, 9
- three Cardan angles, in radian
- a vector and an angle, in radian
- the rotation matrix
- Quaternion (QUATERNION) : 3, 4
- a vector. In this case, a pure quaternion is created
- the 4 components of a quaternion
- PV (QSW or TNW) : 6 components (x, y, z, vx, vy, vz)
For each type, the deviation is computed as per :
- Double : relative deviation
- Vector : euclidian distance
- Angle : angle between the rotations built out of each angle and the same axis
- Date : number of seconds between both dates
- Rotation : angle of the shortest rotation between both rotations
- Quaternion : angle of the shortest rotation between the rotations formed from the normalized quaternions
- PV : deviation of the position and velocity vectors in LOF frame : "QSW" or "TNW". The origin of the LOF frame is the object of FILE1 file)
More details about what is created can be found in the Javadoc of the Facade associated with each type, either in the public constructor for Double, Vector and Angle types, or in the factory methods for the others.
Calling the Ephemeris Comparator
The user can call the Ephemeris Comparator Tool in two different ways :
- by calling the method "EphemerisComparatorMain" in the command line interface :
- Usage : EphemerisComparatorMain [IHM|BATCH]<Ressource file><Input file 1><Input file 2><Output file> [<XSL file>]
- <Ressource file> : absolute path of resource file to use
- <Input file 1> : absolute path of the first file to compare
- <Input file 2> : absolute path of the second file to compare
- <Output directory> : absolute directory of the synthesis files
- <XSL file> : absolute path of the xsl file used in the report buiding to produce a formatted html output
- Usage : EphemerisComparatorMain [IHM|BATCH]<Ressource file><Input file 1><Input file 2><Output file> [<XSL file>]
Note : Only the BATCH option is available at this time
- by calling the method "compareFiles" (class EphemerisComparator) with the following parameters :
- URI setupFileURI : the address to the resource information file
- URI firstDataFileURI : the address of the file that contains the first data set
- URI secondDataFileURI : the address of the file that contains the second data set
- URI outputDirURI : the address (directory) where to put the reports
- URI xslFileURI : the address of the xsl file used in the report buiding to produce a formatted html output
- boolean smartComparison : tells whether to create meaningful object and compare them or just compare raw double values
Note that this function could be called directly in the code, and returns a sum-up of the comparison : a boolean value that is true if no deviation exceeded the associated threshold. This could be used to automate some validation.
Getting Started
TBD
Tips & Tricks
TBD
