Jump to content
PDS Geosciences Node Community

FrankM

Members
  • Posts

    15
  • Joined

  • Last visited

Posts posted by FrankM

  1. Elyse,

     

    I don't see an easy way to bypass clicking OK on the parameter selection dialog. As you suggest, sumpar_widget_event is only called when an event occurs - like pressing OK on the selection dialog.

     

    Maybe you can bypass the entire select_summary_params call though if you know ahead of time which parameters you want to compute. The key input to select_summary_params is summary_list, which is just a string array containing names of all the parameters. Create an integer array containing the indices (in summary_list) of all the parameters you want to compute. Use that in as selected_params which is the important output from select_summary_params.

     

    Here's an example with a shorter-than-reality list of parameters:

    summary_list = ['BDI1000IR', 'OLINDEX3', 'R1330', 'BD1300', 'LCPINDEX2']

    Let's say you want BDI1000IR and R1330 only. Then:

    selected_params = [0,2]

    And you can set that manually (or with code of your own) and skip the select_summary_params call entirely.

     

    Hard coding the indices may be risky. I think the summary_list array will always be the same for a given detector with the current CAT code, but may change with updates. So you may want to insert some simple IDL code to get indices for desired parameters vs. hard coding them, if you want the code to survive very long.

     

    Frank

  2. Hi Elyse,

     

    I verified that the lines in projection_event.pro do pull latitude and longitude from the DDR file. The 'pos' keyword to envi_get_data is zero-offset - so the first band is pos=0 etc. The two lines you quote pull the 4th and 5th bands - lat and lon - as they should. I think the walkthrough slides and probably the Build GLT tool are numbering the bands starting at 1, so 4 & 5 in the slides is really the same as 3 and 4 in the code.

     

    I think the DDR search should find the DDR for your file if it's located in the same directory, despite the filename additions, as long as the root 'frt00003e12_07_if166l_trr3..' matches. I think you would see a failure if it didn't find a DDR; it wouldn't generate results that differed from expected.

     

    How big is the difference you see? Tiny differences might be explained by roundoff, different calculations etc. I can dig deeper if the difference is significant.

     

    Frank

  3. CRISM is getting close to 100,000 total observations of all types. We've acquired close to 16,000 FRT (plus roughly 2000 HRS and 4000 HRL) and a bunch of FRS and ATO that I haven't counted yet. So, I don't know why the number 2900 came up. Probably has something to do with their study area or target.

     

    Frank

  4. As you may know, the tiles are created by compiling multiple, independent observations. Each of the long strips across a tile is a separate observation.)

     

    Some tiles have more coverage that others, and the one you are looking at is more sparse than most. That may be due in part to limitations on CRISM's ability to get coverage over Gale crater, since MRO is frequently conducting relay for MSL there. That prevents CRISM observations to reduce radio interference with the relay. Also, it's fairly low latitude where orbits crossing a given point are less frequent than at higher latitudes.

     

    The browse products for the MRDR images are all generated from each observation, and go into the tile without adjustment. Some products that may be sensitive to seasonal, atmospheric, or photometric variability can show significant variation between observations, and that will show in the tile display as strip-to-strip inconsistency. Some products may not be as sensitive to such variability and might seem more consistent. For example, the blue wavelengths that go into R440 are probably more sensitive to atmospheric scattering than the IR, so dust aerosol variation between observations might explain some of the difference between R440 and IRAC.

  5. Colin,

     

    You need to re-stretch the CIRRUS output. It contains very large values at bad pixels. In the display window for the CIRRUS output go to  Enhance/Interactive Stretching (like before). First go to "Options" in the window that pops up and change the histogram parameters to reasonable values (maybe 0 to 1, you can experiment with this) and then reset the Stretch limits to something reasonable (experiment here too). Do that for all three RGB channels.

     

    Any time you get an all-black output image in the CAT/ENVI processing, restretching the image is a good first thing to try. If you really have all zeros there's a problem, but usually it's just ENVI scaling to 65535 or a hot pixel or something.

     

    As for the striping, there is frequently a little minor striping in the summary parameters. I'm not sure why it's showing up in the CAT and not in the published browse image, unless the resolution of the png is just not enough to capture it.

     

    Frank

  6. Here's how you can reproduce the published browse image.

     

    First, get some info about the published browse image. If you're looking at the products on crism-map.jhuapl.edu, click on the "Map/Stretch info" link in the box with the ir_maf thumbnail image. Note the ranges on the RGB channels (0-0.13, 0-0.1, 0-0.2)

     

    Now in ENVI with CE1D, do what you did, which produces the autoscaled display with the bright colors.

     

    Then from the display window with that image, go to Enhance/Interactive Stretching...

     

    A window pops up; note the R-G-B radio buttons; R should be selected initially, that's fine, now go to the "Options" menu, click "Histogram Parameters" and enter the Red channel min and max (0 and 0.13 from the map/stretch info data you looked at before) in the Histogram Min and Histogram Max boxes, and click "Apply." Now go back to the Histogram Stretching window and select the "G" radio button, then go back to the parameter box, enter the G channel min (0)  and max (0.1), and click apply. Now repeat for blue. When you're all done, close the little histogram parameter box. Then finally click "Apply" in the Interactive Strecting window, and the display colors should restretch to match the published image.

     

    Maybe there's an easier way - but that's the procedure I know.

    Frank

  7. I think you'll always get the gigantic spike at ~2.7 microns, and the reason is the same - there are pixels set to 65535 to flag bad bands there, and that flag value is carried through all the processing steps. If you manually rescale the spectral plot it should look OK at other wavelengths.

     

    You may also get smaller spikes (few percent of continuum) in the 1.9-2.1 micron range because of artifacts from the volcano scan correction. The empirical volcano scan optimization is supposed to minimize these artifacts. You might want to experiment with using different volcano scans (you can select your own with the "User-selected volcano scan" option), but this should not have much effect away from the region right around 2.0 microns.

  8. Hi Colin,

     

    First, about the big spike at 2.75 microns:

     

    There are bad bands in the CRISM data in that region that are filled with "65535." When autoscaled that obviously drives the y axis way out of range so that you don't even see the actual I/F data, which is all normally < 1.

     

    There are two z-profile menu items available in CAT/ENVI on the display window Tools menu. The one called "Z Profile (Spectrum)" is the native ENVI routine, and it does not recognize the 65535 as a bad value flag (even though it's specified in the header). So that spectral plot ends up dominated by the spike, as you describe. You can fix that as follows. In the Edit menu on the spectral plot, go to "Plot Parameters...", click the "Y-Axis" radio button, and then enter a reasonable value in the second "Range" box (that should start out filled with 65535). You can usually start out with 0.5 or so and adjust from there. Hit "Apply" at the bottom to adjust the plot.

     

    The other Z profile tool is a custom CRISM one called "**CRISM**CDR Z Profile..." and it ignores the 65535 values and does a better scaling on the y axis. However, it does not update as you interactively move around the image like the ENVI profile does. You get a plot for one pixel, and then if you want to see another pixel you have to move there and do another profile.

     

    Second, about the wavelength axis labeling order:

    Maybe you're plotting the spectral profile from the wrong display here. You may have multiple display windows open, some showing CAT-converted data, and some showing the original CRISM data which is not swapped. Each window has its own Tools/Z Profile command, which will pull a spectrum from the file that window is displaying. So even if you've done the CAT conversion, if you go to the Tools menu on a display for the raw CRISM data, you get a spectrum with reversed wavelengths. Make sure you use the Tools menu on the display for the CAT-converted data. It should plot wavelengths in the order you want.

     

    One note on that - the spectral plot with reversed wavelengths is not exactly wrong, the spectral data is reversed along with the axis labels; so it's technically accurate, even if it's not what you ultimately want. At least it should be. You can check by looking for a couple of characteristic, ubiquitous features. You should see a narrow "triplet" feature at 2.0 microns when you haven't done a volcano scan correction. And going from shorter to longer wavelengths, the spectrum should dive at about 2.7 and then ramp up between there and 3.7 or so microns. If the axis labels don't match up with those features there is a more serious problem.

     

    And one more general note. You mention not doing any further corrections because the TRR3 data is already cleaned. You probably should run a volcano scan atmospheric correction to remove the CO2 feature at 2 microns, unless you're studying that atmospheric feature. That's not removed in TRR3 TRDR data. But you don't generally need to run the data filtering utilities.

     

    Frank

     

  9. Hi Mahima,

     

    That's commonly caused by not setting up the envi.cfg file correctly. In the CAT distribution there are files envi_win.cfg and envi_unix.cfg (in CAT_ENVI directory). In your case (running Windows) you should copy envi_win.cfg to envi.cfg. If there's not a file called envi.cfg ENVI will not create the CAT_ENVI menu.

     

    If you did create the envi.cfg file, the next thing to check is the IDL path. Try the command:

    IDL> print,!PATH

    the path should include several directories with Program Files/CAT_ENVI/...

    If you don't see that ENVI probably won't find the files it needs to generate the CAT_ENVI menu. Try following the IDL path instructions again, and make sure you do the permanent path set method.

     

    Let me know if neither of these is the problem and we can try some further troubleshooting.

     

    Frank

  10. Hi Elyse,

    There's no list of parameter thresholds available for CRISM. For most parameters something in the 0.05-0.1 range is probably about right. Zero may be appropriate for some of the BD parameters, or bump up to 0.005 for noisy scenes.

     

    The Poulet et al thresholds may be pretty good for most of the CRISM multispectral parameters in the current CAT (up to 7.1), since the CAT parameters are mostly defined parallel to the OMEGA parameters.

     

    Revisions to CRISM parameters are in development and appropriate thresholds for the updated formulations are a subject of active research.

     

    Depending on what you're doing, you may not want to put too much emphasis on thresholding. Can be useful for statistical analysis or sifting through large amounts of data searching for something interesting, but once you focus on a particular observation a hard threshold might obscure interesting exposures.

    Frank

  11. Hi Elyse,

     

    From my experience, I would say values ~0.04 are marginal olivine indications for that parameter. Values closer to 0.1 are more robust but still could be an artifact. In particular there can be problems in shadows and on extreme albedos for the mafic paracmeters.

     

    With values in that ~0.04 range for olindex you should definitely check the spectra to make sure there's a real olivine signature. You should see broad absorption around 1.0 micron in both VNIR and IR. It's not a bad idea to check the spectra in any case, even with relatively large summary parameter values.

     

    If you're looking at hyperspectral data (FRT, HRL, etc) you can also check the hyperspectral summary parameter OLINDEX2. It avoids some of the problems of the mutlispectral parameter.

     

    For the pyroxenes, values near ~0.05 tend to be more robust than for olivine I think. They are quite susceptible to problems in shadows though.

     

    Frank M

     

  12. Jun,

    There is a bug in one of the IDL routines used for destriping that's causing the problem with that observation. I'll get a patch into the next CAT release.

     

    In general you should be able to process TRR3 CRISM data without additional destripe/despike, since a noise filter has already been applied in the pipeline processing. Those routines were much more important for the old TRR2 calibration version. There may still be noisier-than-usual TRR3 data where additional filtering is beneficial, but you shouldn't have to routinely use the CIRRUS filters with TRR3 data.

     

    I will patch the bug though. Thanks for pointing it out to us.

     

    Frank

     

  13. Hi Jun,

     

    I'm working on this problem. I'll post an updated CAT when I get it fixed.

     

    In the meantime you can work around this issue by omitting the CIRRUS destripe/despike routines for cases where they fail. The TRR3 version of the calibration has some filtering built in so these routines aren't as crucial as they used to be.

     

    Sorry for the delay though, I'll try and get them working as quick as I can in case your data really needs the additional filtering.

     

    And about the volcano scan - the "empirically optimized for this observation" choice should usually be the best. However, if the 2 micron region is critical and you see spiky or bowl-shaped artifacts there you can try some of the other methods and see if other VS work better.

     

    Frank

     

  14. EJA,

     

    An absorption-like artifact on the order of a few percent of the continuum level is typical after applying the volcano scan as currently implemented in CAT.

     

    We've developed a method for correcting that artifact that will appear soon in CAT 6.8.

     

    The volcano scan derives an atmospheric transmission spectrum from the base/summit ratio of images over Olympus Mons, scales it, and divides it out of the scene you're correcting to try to remove the atmospheric CO2 signature. However, the shape of the co2 absorption spectrum varies with altitude, so the volcano scan ratio of a low/high altitude spectrum isn't shaped quite like the observed spectrum at any altitude. That error in the transmission spectrum causes the imperfect correction that leaves the feature you're noticing.

     

    The new method applies the usual volcano scan, and then scales and adds in a "patch" derived from the volcano scan data. In testing so far it helps most of the time. It only seems to hurt when there's CO2 ice in the scene; that throws off the scaling and corrupts the ice spectrum (it's still there, but the shape is messed up).

     

    More details with the CAT 6.8 release. That release should come out in June.

×
×
  • Create New...