Jump to content
PDS Geosciences Node Community

Susie Slavney

Geo Staff
  • Posts

    213
  • Joined

  • Last visited

Posts posted by Susie Slavney

  1. There is a one-to-many mapping from dataless SEED files (and StationXML files) to the miniSEED files (and GeoCSV files). In the PDS label for each miniSEED file there is a pointer to the dataless SEED that was delivered with it. Look for the attribute <insight:metadata_file_name>.  The dataless SEED files are cumulative, and you are encouraged always to use the latest ones, even if the label indicates an earlier version.

    The collection inventory is not guaranteed to be in any particular order, although typically new files are added to the bottom with each delivery.  

  2. Hi Kevin,

    The file naming scheme is documented in the InSight SEIS Software Interface Specification section 4.1.3.1 (https://pds-geosciences.wustl.edu/insight/urn-nasa-pds-insight_documents/document_seis/seis_sis.pdf). The number you indicate in the file name is the revision of the miniSEED file. This is the revision internal to the SEIS science team, as they may go through several revisions before the data file is delivered to PDS. If we receive multiple products for which everything in the file name is the same except this revision number, we consider them to be separate products and we archive them all.

    You ask how to know the revision number ahead of time. You'll find a list of all the products in a collection in the collection inventory. For the SEIS data_seed collection, which includes the SEED and miniSEED files, the collection inventory is here: 

    https://pds-geosciences.wustl.edu/insight/urn-nasa-pds-insight_seis/data/collection_data_seed_inventory.csv

    As you may know from reading the documentation, every SEED or miniSEED file has an equivalent ASCII text file. The ASCII-equivalent file for a SEED product is a StationXML file. The ASCII-equivalent file for a miniSEED product is a GeoCSV file. These are in the data_table collection, and its collection inventory is here:

    https://pds-geosciences.wustl.edu/insight/urn-nasa-pds-insight_seis/data/collection_data_table_inventory.csv

    In these inventories, you'll see that each product is listed by its unique Logical Identifier with Version Identifier (LIDVID), like this:

    P,urn:nasa:pds:insight_seis:data_seed:xb.elyse.03.bhu.2019.042.11::1.1
    The "P" means this product is a primary member of the collection. You can ignore it. The final "::1.1" is the Version Identifier (VID). This is the PDS version number, which is unrelated to the SEIS internal revision number discussed above. The first PDS version of every product is 1.0. In this example, 1.1 indicates that there has been a minor revision, probably a change to the PDS label. 

    The inventory does not include the directory path to each file, but you can figure it out. For example, from the product identifier xb.elyse.03.bhu.2019.042.11, you can assume that product is found in the directory data/xb/continuous_waveform/elyse/2019/042/. Both the miniSEED file and the ASCII-equivalent file are in that directory, along with their PDS labels in XML files. 

    I hope this helps.

    Susan Slavney

     

  3. Jee,

    1. Yes.

    2. To assign each pixel a latitude and longitude, you will need the information in the IMAGE_MAP_PROJECTION part of the label, and also the file DSMAP.CAT in the CATALOG directory of the archive (or DSMAP_POLAR.CAT if you are working with an image in polar stereographic projection). 

    DSMAP.CAT is a text file that gives the details of the simple cylindrical map projection, including the equations to go between latitude-longitude and line-sample. The IMAGE_MAP_PROJECTION keywords in the label give you the values to plug in to those equations for that particular image. 

    3. The data are in 16-bit signed integers because some of the values are negative. Even after applying the scale and offset there will be negative values, representing elevation below the defined planetary radius, as explained in the note in the label. 
     

  4. Jee,

    The IMG files are just binary arrays, nothing complicated. The label file that accompanies each image file gives the information your software needs to read the file. For instance, in LDEM.LBL in the IMAGE object, you'll see these lines:

        LINES                 = 720
        LINE_SAMPLES          = 1440
        MAXIMUM               = 21008
        MINIMUM               = -17758
        SAMPLE_TYPE           = LSB_INTEGER
        SAMPLE_BITS           = 16
        UNIT                  = METER
        SCALING_FACTOR        = 0.5
        OFFSET                = 1737400.
    

    This tells you the image array has 720 lines with 1440 samples (pixels) in each line. (The pixels should be displayed from left to right, top to bottom.) Each pixel is a 16-bit signed least-significant-byte-first (little-endian) integer. To go from pixel value to meters, multiply the pixel value by 0.5 and add 1737400. The range of values in the array is from -17758 to 21008 meters. 

    See also Frequently Asked Questions About LOLA Data.

     

     

  5. As June says, the data values are int16, signed 16-bit integers. I think your fread statement looks correct, although I am not a Matlab user. The PDS3 label says SAMPLE_TYPE = LSB_INTEGER, which is defined as a signed integer in the PDS3 data dictionary, although it would have been helpful if the label explicitly declared the samples to be signed. You may assume that LSB_INTEGER and MSB_INTEGER in PDS3 labels always mean signed integers. LSB_UNSIGNED_INTEGER and MSB_UNSIGNED_INTEGER are used when the values are unsigned.

  6. MISSING_CONSTANT is the value used to indicate that no actual data value exists; that is, the data value is missing. It does not make sense to apply the SCALE and OFFSET to these pixels. Pixels with this value should be omitted from computations. The value that indicates "missing data" is chosen to be outside the expected range of actual data values. 

    The PDS3 Data Dictionary defines MISSING_CONSTANT as follows: "The missing_constant element supplies the value used to indicate that no data were available." (https://pds.nasa.gov/datasearch/subscription-service/SS-20200212.shtml). Examples of its use are given in the PDS3 Standards Reference (https://pds.nasa.gov/datastandards/pds3/standards/sr/StdRef_20090227_v3.8.pdf). 

  7. The SKYV* files are in the EXTRAS directory, which means they are not part of the official LOLA PDS archive, and therefore not peer reviewed. They are extra material that users may find interesting or helpful as an addition to the standard data products. 

    For the product you mentioned, SKYV_65N_240M, there is an equivalent IMG product in http://imbrium.mit.edu/EXTRAS/ILLUMINATION/IMG/. According to its label, it is a binary array of 6420 by 6420 16-bit LSB signed integers (LSB = least-significant-byte-first = "little-endian"). To go from the DN value to solid angle in steradians, you apply the formula DN*SCALING_FACTOR+OFFSET, where SCALING_FACTOR is 0.0002 and OFFSET is 6.2832. 

  8. Hi,

    I don't see any product in the LOLA archives with the name SKYV_65N_240M_JP2. Can you give me the URL where you found it?

    In general, the JP2 files are JPEG2000 images. You won't be able to read them easily in MATLAB. 

    In the LOLA archives, JP2 images are included only for ease of displaying the data. For every JP2 file there is a corresponding IMG file containing the same data in plain raster format, which is readable in MATLAB. The OBJECT = IMAGE section of the label (*.LBL file) tells you the information MATLAB needs to know to read the file, such as number of lines, number of samples per line, size of a sample, and sample type (byte order).
     

  9. We were contacted by a user trying to read SHARAD EDR data with ReadPDS3 for IDL, but that tool does not read the bit fields and three-byte integers in the data product, and as it is a PDS3 tool it is not likely to be developed further.

    We suggested the following tools for working with SHARAD data.

    There was a SHARAD/MARSIS workshop at LPSC in 2014. Presentations are on the Geosciences Node’s web site at https://pds-geosciences.wustl.edu/workshops/sharadmarsis_Mar14.html. There's one by Than Putzig about the CO-SHARPS Processing Boutique, software that reads SHARAD products (not clear if he means EDRs, RDRs, or both) and creates radargrams, among other things. To get the software you have to request it from PSI at https://www.psi.edu/SHARAD.

    There are links to other software packages on the workshop page, but nothing specifically for reading EDRs. Most people prefer to use the RDRs. 

    We have also posted an IDL script for generating browse images on the SHARAD page (https://pds-geosciences.wustl.edu/missions/mro/sharad.htm), as an example of how to read an RDR product. It might be possible to modify the code to read an EDR.

    Finally, we have some C# code to read the SHARAD EDRs, but the code does not read every field in the *.dat file. If you can let us know which fields you are interested in, we can try to modify the code to see if it works.

     

    The user replied, "for now that's fine for us skipping the unread fields. We could read the other data columns by slightly modifying the label/fmts of the SHARAD EDR. In case we would have to actually use those specific fields, I think that we could implement some specific reading routines, but it is some time to spend we don't really have for a non-critical purpose."

     

     

  10. Some users of LRO Diviner data have asked why there are some data products with negative temperature values. For example: "I'm exploring the temperature data of the Diviner, I have downloaded TB (Brightness Temperature) data for 6,7,8, and 9 channels and I found negative values (in K) in a few of the images after conversion (TB= (DN * scaling factor) + Offset). This negative value is not possible in K, I'm unable to understand this."

    Answers from Diviner team members:

    "The shorter wavelength channels are noisy at low temperatures. Each channel has a minimum temperature below which it becomes noisy. These minimum temperature cutoffs go down with longer wavelength channels, so channels 8 and 9 are really the only ones that return useful data in the PSRs. When the channel gets below this minimum temperature, it starts returning negative radiance values. This would explain why channel 4 is giving you negative temperatures, while channel 9 does not. I've attached the Supporting Material from the Paige et al 2010 Science paper [see attachment Paige.SOM.pdf]. The table gives the minimum temperatures for the IR channels (3-9)." (Dr. Jean-Pierre Williams, UCLA, 2019-03-11)

    "The negative temperatures are actually "good" data. When the net signal at the detectors after calibration gets close to zero, Diviner records both small  positive and small negative radiances. When the negative radiances are turned into brightness temperatures, they are reported as brightness temperatures. Zero-ing out the negative radiances would be completely inappropriate, because it is possible to pull signal out of the noise by averaging. You should see more negative temperatures in the shorter wavelength channels (3-6), as well as all channels in the coldest places near the poles.

    When the thermal emission from the moon is so small that there is no measurable signal, there's a 50% probability that a small negative signal will be measured. We convert these to brightness temperatures, and flag them as negative brightness temperatures, with the understanding that negative temperatures are not possible.

    In isolation, you can't convert negative temperatures into meaningful temperatures.  What we generally do is pick a threshold below which all temperatures should be ignored. This will vary depending on which Diviner channel you are using." (Dr. David Paige, UCLA, 2020-03-24)

    Paige.SOM.pdf

  11. Many publishers of scientific research, including AGU, are now requiring authors to make their data available in a public repository as a condition of publication. (See AGU's policy here.) The Geosciences Node has been contacted by a few such authors asking whether the PDS will receive their data so that their articles can be published. In some cases the data set in question is a suitable candidate for a PDS archive, but usually it is not. 

    If you are the author of an article for a journal that requires you to archive your data in a public repository, you may not wish to undertake the steps involved in submitting the data to the PDS, steps that include labeling, documenting, and possibly reformatting the data products. The effort involved may not be justified for a small, simple data set. In this case we recommend that you submit the data to one of the many available online data repositories, such as figshare (figshare.com) or Dataverse (dataverse.org).

    On the other hand, if you believe your data would be a useful addition to the PDS, and you are willing to put in the work and submit the data to a peer review, please send your request to geosci@wunder.wustl.edu

  12. From www.mars.asu.edu/data you can download the TES mineral maps with the raw data in VICAR format. GDAL (https://www.gdal.org/) can read and export VICAR files. For example, these GDAL commands work on the VICAR files:

    gdalinfo   TES_Glass_Clay_numeric.vic

    gdal_translate   -of   TES_Glass_Clay_numeric.vic    TES_Glass_Clay_numeric_gdal.tif

     

    For another resource, there is MATLAB code to read VICAR files on Github at https://github.com/jnulzl/vicarRead

×
×
  • Create New...