SDSS Spectroscopy Manual Inspection

In this document:

Related documents:

Overview

Currently, the new reductions exist at Princeton in:

  spectro.princeton.edu:/u/dss/spectro_test
The goal is to have at least 2 trusted people inspect each of the 388 DR1 plates.

Inspection will be done using plain text files in a central repository where all SDSS folks have access. Template files have been auto-generated with the redshifts and classifications from the pipelines. Alternatively, Michael's SM scripts or my IDL code can be used to generate these simple files on-the-fly.

When someone inspects a plate, these files are edited and imported back into this repository.

CVS "spinspect" Product

There is a new product called "spinspect", which can be downloaded (checked-out) or modified (checked-in) by a list of users that includes the "alfred" account at Princeton. If you have access from your Princeton account:

  % bash
  % CVSROOT=sdsscvs.astro.princeton.edu:/usr/local/cvsprivate
  % CVS_RSH=ssh
  % cvs co spinspect
If you have access through the "alfred" account:
  % bash
  % CVSROOT=alfred@sdsscvs.astro.princeton.edu:/usr/local/cvsprivate
  % CVS_RSH=ssh
  % cvs co spinspect
Once you have this product in your home directory, you can update with other people's changes with:
  % cd ~/spinspect
  % cvs update

The template files can be found in

  spinspect/prototype/spInspect-$plate-$mjd.par
Once a plate has been inspected, that data is imported in files
  spinspect/data/$inspector/spInspect-$plate-$mjd-$inspector.par
Currently, only folks with a Princeton account and belonging to group dss have check-in permission for this product. Users from the alfred account only have check-out access, and will have to ask someone else to check-in their files -- sorry, but this is a computer security issue.

There is also a mechanism for folks to submit a random set of objects from multiple plates. Such a list can be put in files by the name:

  spinspect/data/$inspector/spInspect-0000-00000-$inspector.par

From these files, we will be able to auto-generate files/web-pages with wrong redshifts or interesting objects, and compare different people's ID's. We can even auto-generate the contradiction lists and ask folks to resolve conflicting ID's.

Importing Into "spinspect" Product

If this is your first import and your name is "knapp", you need to create your directory and add it to the "spinspect" product:

  cd spinspect/data
  mkdir knapp
  cvs add knapp
  cvs commit -m "Creating my data directory" knapp
Once you put your files in this directory, they can be imported as follows:
  cd spinspect/data/knapp
  cvs add spInspect-0400-51820-knapp.par
  cvs commit -m "I have inspected this" spInspect-0400-51820-knapp.par

The Rules of Inspection

  1. The Yanny header should be modified to list your name as the "inspector", and the file name should also contain your name (lower-case, please).
  2. The fields plate,mjd,fiberid,class,subclass,z are from the 2D+1D code corresponding to the version numbers in the header of the file. Sky fibers will always be called "SKY". Non-sky objects with the ZWARNING flag set will be called "UNKNOWN". For both the case of SKY and UNKNOWN, the best-fit redshift is put in the "z" field just for convenience.
  3. No entries in the "manual" fields is an implicit declaration that the ID is correct.
  4. The manual_class field can have the following values:
  5. The value manual_z=0 is special, and means the redshift should be unchanged. If you need to change the redshift, it should be set to some non-zero value, even if that is 0.0000001.
  6. If the redshift appears to be correct, then the manual_comments field could be edited. This allows anyone the option to use agreed-upon comments or their own codes for tracking types of objects, i.e. "E+A" for E+A galaxies. See Adding Comments section for more details.
  7. The manual_subclass field is mostly used for sub-classifying stars.
  8. Deleting an object from this file is equivalent to saying CLASS="?", e.g. you either have not inspected the object or do not know what it is.

Syntax Rules

The rules for these Yanny files are as follows:
  1. Empty fields should be denoted by double-quotes.
  2. Strings that contain whitespace should be enclosed in quotes (but can be in quotes even if a single word.)
  3. Strings should not contain either single- or double-quotes within the string.
  4. Non-quoted whitespace is ignored.
  5. Lines can be very long, or can be continued with a backslash, even within a quoted string.

Adding Comments

The "comments" field should mostly use agreed-upon words. These words should be case-sensitive. Multiple comments should be semi-colon separated. There is no point in adding comments like "Carbon star" when an object is already classified as such (if it is not, then the manual_subclass field should be edited).

Adding Long Comments (Text)

Long comments should be put in the SPECTEXT structure, as in the example below. There are no restrictions on this text, other than it conform to the syntax rules.

Superpositions of Objects

Superpositions of objects can be identified by putting information for the dominant object in the SPECINSPECT structure, and any secondary objects in SPECBLEND structures. See example below.

Problems in the Spectra

The following set of problems are well-known. Problem reports already exist for these problems, but you could reference the PR numbers in the "comments" field of the inspection files.

  1. Some plates are just down-right bad. These are excluded from the DR1 plate list. More details of bad plates can be found on the Plate Quality List or at http://spectro.princeton.edu/#problems_known
  2. Spectro-photometry can have coherent +/- 5% errors, especially near the dichroic (PR 2528). For each plate, these errors can be seen in the plots "spDiag1d-$plate-$MJD.ps". There is no point in commenting on these errors, unless they result in the mis-classification by the pipelines.
  3. Very occasionally, a small wavelength-calibration error results in bad sky-subtraction (PR 3399). Note that the BRIGHTSKY mask bit should be set in these cases.
  4. Galaxies with redshifts >0.6 not found (PR 3155). The P-1D pipeline currently does not search for galaxies at high redshift.
  5. Some good QSO fits are incorrectly rejected with the NEGATIVE_EMSSION bit set in the ZWARNING flag (PR 3797). Examples are plate 400/51820, fibers 21, 512, 632.

New problems should be filed in the Bug Database against the idlspec2d product for problems in the spectra, or against specBS for problems in the classifications, redshifts, or other measured quantities.

Example Files

For plate 400/51820, I edited the following 5 lines in the file "spInspect-0400-51820-schlegel.par":

SPECINSPECT 400 51820 21 UNKNOWN BROADLINE 1.44154 schlegel "QSO" "" 1.44154 "PR 3797"
SPECINSPECT 400 51820 193 UNKNOWN BROADLINE 0.997749 schlegel "QSO" "" 0.997749 "PR 3797"
SPECINSPECT 400 51820 512 UNKNOWN BROADLINE 0.399743 schlegel "QSO" "" 0.399743 "PR 3797"
SPECINSPECT 400 51820 632 UNKNOWN BROADLINE 1.49835 schlegel "QSO" "" 1.49835 "PR 3797"
SPECINSPECT 400 51820 640 UNKNOWN "" 0.442294 schlegel "GALAXY" "" 0.442294 "Badskysub; PR 3399"
Below is the beginnings of a fake example file, with entries for superpositions, etc:
plate 400
mjd 51820
vers2d v4_9_8
verscomb v4_9_8
vers1d v4_9_8
inspector knapp

typedef struct {
  int plate;
  int mjd;
  int fiberid;
  char class[40];
  char subclass[40];
  float z;
  char inspector[20];
  char manual_class[40];
  char manual_subclass[40];
  float manual_z;
  char manual_comments[80];
} SPECINSPECT;

typedef struct {
  int plate;
  int mjd;
  int fiberid;
  char inspector[20];
  char manual_text[800];
} SPECTEXT

typedef struct {
  int plate;
  int mjd;
  int fiberid;
  char inspector[20];
  char manual_class[40];
  char manual_subclass[40];
  float manual_z;
  char manual_comments[80];
} SPECBLEND

specinspect 400 51820 1 GALAXY  "" 0.100000 knapp GALAXY "" 0.200 "Blend: galaxy+star"
specinspect 400 51820 2 UNKNOWN "" 0.123456 knapp "" "" 0 ""
specinspect 400 51820 3 SKY     "" 0.654321 knapp "" "" 0 ""
specinspect 400 51820 4 STAR    F2 0.000010 knapp "" G5 0 ""
specinspect 400 51820 5 GALAXY  "" 0.100000 knapp ? "" 0 ""
specinspect 400 51820 6 GALAXY  "" 0.100000 knapp UNKNOWN "" 0 "Artifact; Glitch at 5000 Ang"
...

specblend 400 51820 1 GALAXY  "" 0.100000 knapp STAR M 0.00001 ""

specinspect 400 51820 6 knapp "This is the craziest \
  object that I have ever seen in my life.  Someone \
  should really use a few nights of Keck time and see."

David Schlegel, schlegel@astro.princeton.edu