SDSS Spectroscopy Manual Inspection
In this document:
Related documents:
Currently, the new reductions exist at Princeton in:
spectro.princeton.edu:/u/dss/spectro_test
The goal is to have at least 2 trusted people inspect each of the 388
DR1 plates.
Inspection will be done using plain text files in a central repository where
all SDSS folks have access. Template files have been auto-generated with
the redshifts and classifications from the pipelines. Alternatively,
Michael's SM scripts or my IDL code can be used to generate these simple
files on-the-fly.
When someone inspects a plate, these files are edited and imported back
into this repository.
There is a new product called "spinspect", which can be downloaded
(checked-out) or modified (checked-in) by a list of users that includes
the "alfred" account at Princeton. If you have access from your
Princeton account:
% bash
% CVSROOT=sdsscvs.astro.princeton.edu:/usr/local/cvsprivate
% CVS_RSH=ssh
% cvs co spinspect
If you have access through the "alfred" account:
% bash
% CVSROOT=alfred@sdsscvs.astro.princeton.edu:/usr/local/cvsprivate
% CVS_RSH=ssh
% cvs co spinspect
Once you have this product in your home directory, you can update with
other people's changes with:
% cd ~/spinspect
% cvs update
The template files can be found in
spinspect/prototype/spInspect-$plate-$mjd.par
Once a plate has been inspected, that data is imported in files
spinspect/data/$inspector/spInspect-$plate-$mjd-$inspector.par
Currently, only folks with a Princeton account and belonging to
group dss have check-in permission for this product. Users from
the alfred account only have check-out access, and will have to
ask someone else to check-in their files -- sorry, but this is
a computer security issue.
There is also a mechanism for folks to submit a random set of objects from
multiple plates. Such a list can be put in files by the name:
spinspect/data/$inspector/spInspect-0000-00000-$inspector.par
From these files, we will be able to auto-generate files/web-pages with wrong
redshifts or interesting objects, and compare different people's ID's.
We can even auto-generate the contradiction lists and ask folks to
resolve conflicting ID's.
If this is your first import and your name is "knapp", you need
to create your directory and add it to the "spinspect" product:
cd spinspect/data
mkdir knapp
cvs add knapp
cvs commit -m "Creating my data directory" knapp
Once you put your files in this directory, they can be imported as follows:
cd spinspect/data/knapp
cvs add spInspect-0400-51820-knapp.par
cvs commit -m "I have inspected this" spInspect-0400-51820-knapp.par
- The Yanny header should be modified to list your name as the "inspector",
and the file name should also contain your name (lower-case, please).
- The fields plate,mjd,fiberid,class,subclass,z are from the 2D+1D code
corresponding to the version numbers in the header of the file.
Sky fibers will always be called "SKY". Non-sky objects with the ZWARNING
flag set will be called "UNKNOWN". For both the case of SKY and UNKNOWN,
the best-fit redshift is put in the "z" field just for convenience.
- No entries in the "manual" fields is an implicit declaration that the ID
is correct.
- The manual_class field can have the following values:
- GALAXY
- QSO
- STAR
- SKY
- NOOBJECT - There is no object at this position on the sky, which
can happen for example if a satellite track or blead trail
has been targetted.
- UNKNOWN - Objects not identifiable from their spectra should
have this value. This includes spectra with bad artifacts, in
which case the word "Glitch" or "Artifact" should be included in the
comments field. If an object is truly not identifiable from
the spectrum, then "UNKNOWN" **is** the correct answer.
If you cannot classify this object, but think it is some
truly interesting unclassifiable object, then the word "Weird"
or "TrulyWeird" should be put in the comments field.
- *? - If you don't have the confidence that an ID is correct,
then put "?" in the manual_class field. This will be interpreted
to say that you have not inspected that particular object or have
low confidence. You could use "GALAXY?" or just "?".
- The value manual_z=0 is special, and means the redshift should be
unchanged. If you need to change the redshift, it should be set to
some non-zero value, even if that is 0.0000001.
- If the redshift appears to be correct, then the manual_comments
field could be edited. This allows anyone the option
to use agreed-upon comments or their own codes for tracking types
of objects, i.e. "E+A" for E+A galaxies. See
Adding Comments section for more details.
- The manual_subclass field is mostly used for sub-classifying stars.
- Deleting an object from this file is equivalent to saying CLASS="?",
e.g. you either have not inspected the object or do not know what it is.
The rules for these Yanny files are as follows:
- Empty fields should be denoted by double-quotes.
- Strings that contain whitespace should be enclosed in quotes (but can
be in quotes even if a single word.)
- Strings should not contain either single- or double-quotes within
the string.
- Non-quoted whitespace is ignored.
- Lines can be very long, or can be continued with a backslash, even
within a quoted string.
The "comments" field should mostly use agreed-upon words. These words
should be case-sensitive. Multiple comments should be semi-colon separated.
There is no point in adding comments like "Carbon star" when an object
is already classified as such (if it is not, then the manual_subclass
field should be edited).
- Featureless = featureless spectrum, as so many FIRST sources seem to be
- Weird = The object was classified as UNKNOWN, but you think it's an
interesting object.
- TrulyWeird = This object isn't just weird, but you think it
is a truly interesting object.
- Blend = Superposition of objects; the second object should have
its information put in a SPECBLEND structure
- Supernova = supernova on spectrum
- Artifact = bad artifact in the spectrum, whether or not it should
be classified as UNKNOWN
- Glitch = An artifact that effects only a small number of pixels
- Badskysub = Bad sky-subtraction (probably related to PR 3399).
- PR xxxx = Reference to a Problem Report number, for example "PR 1234".
- BAL = BAL QSO; Pat Hall suggests using any BAL subtypes as long as
they include "BAL" in the string, i.e. HiBAL, LoBAL, FeLoBAL,
mntBAL, otBAL, redBAL, fe3BAL.
- E+A = E+A galaxy
- BLLac = BL Lac
- BEMstar = Dwarf stars w/strong broad cyclotron emission
- MgIIabs = MgII absorption system in the spectrum
- WDmagnetic = Magnetic white dwarf
- NearBrightStar = Scattered light from a bright star,
i.e. 387/51791-185 has a 7th mag K star 25 arcsec to the SE
Long comments should be put in the SPECTEXT structure, as in the
example below. There are no restrictions on this text, other than
it conform to the syntax rules.
Superpositions of objects can be identified by putting information for
the dominant object in the SPECINSPECT structure, and any secondary objects
in SPECBLEND structures. See example below.
The following set of problems are well-known. Problem reports already
exist for these problems, but you could reference the PR numbers in
the "comments" field of the inspection files.
- Some plates are just down-right bad. These are excluded from
the DR1 plate list. More details of bad plates can be found on
the Plate
Quality List or at
http://spectro.princeton.edu/#problems_known
- Spectro-photometry can have coherent +/- 5% errors, especially
near the dichroic (PR 2528). For each plate, these errors can be seen
in the plots "spDiag1d-$plate-$MJD.ps". There is no point in commenting
on these errors, unless they result in the mis-classification by the
pipelines.
- Very occasionally, a small wavelength-calibration error results
in bad sky-subtraction (PR 3399). Note that the BRIGHTSKY mask bit
should be set in these cases.
- Galaxies with redshifts >0.6 not found (PR 3155).
The P-1D pipeline currently does not search for galaxies at high redshift.
- Some good QSO fits are incorrectly rejected with the NEGATIVE_EMSSION
bit set in the ZWARNING flag (PR 3797).
Examples are plate 400/51820, fibers 21, 512, 632.
New problems should be filed in the Bug Database
against the idlspec2d product for problems in the spectra,
or against specBS for problems in the classifications, redshifts,
or other measured quantities.
For plate 400/51820, I edited the following 5 lines
in the file "spInspect-0400-51820-schlegel.par":
SPECINSPECT 400 51820 21 UNKNOWN BROADLINE 1.44154 schlegel "QSO" "" 1.44154 "PR 3797"
SPECINSPECT 400 51820 193 UNKNOWN BROADLINE 0.997749 schlegel "QSO" "" 0.997749 "PR 3797"
SPECINSPECT 400 51820 512 UNKNOWN BROADLINE 0.399743 schlegel "QSO" "" 0.399743 "PR 3797"
SPECINSPECT 400 51820 632 UNKNOWN BROADLINE 1.49835 schlegel "QSO" "" 1.49835 "PR 3797"
SPECINSPECT 400 51820 640 UNKNOWN "" 0.442294 schlegel "GALAXY" "" 0.442294 "Badskysub; PR 3399"
Below is the beginnings of a fake example file, with entries for
superpositions, etc:
plate 400
mjd 51820
vers2d v4_9_8
verscomb v4_9_8
vers1d v4_9_8
inspector knapp
typedef struct {
int plate;
int mjd;
int fiberid;
char class[40];
char subclass[40];
float z;
char inspector[20];
char manual_class[40];
char manual_subclass[40];
float manual_z;
char manual_comments[80];
} SPECINSPECT;
typedef struct {
int plate;
int mjd;
int fiberid;
char inspector[20];
char manual_text[800];
} SPECTEXT
typedef struct {
int plate;
int mjd;
int fiberid;
char inspector[20];
char manual_class[40];
char manual_subclass[40];
float manual_z;
char manual_comments[80];
} SPECBLEND
specinspect 400 51820 1 GALAXY "" 0.100000 knapp GALAXY "" 0.200 "Blend: galaxy+star"
specinspect 400 51820 2 UNKNOWN "" 0.123456 knapp "" "" 0 ""
specinspect 400 51820 3 SKY "" 0.654321 knapp "" "" 0 ""
specinspect 400 51820 4 STAR F2 0.000010 knapp "" G5 0 ""
specinspect 400 51820 5 GALAXY "" 0.100000 knapp ? "" 0 ""
specinspect 400 51820 6 GALAXY "" 0.100000 knapp UNKNOWN "" 0 "Artifact; Glitch at 5000 Ang"
...
specblend 400 51820 1 GALAXY "" 0.100000 knapp STAR M 0.00001 ""
specinspect 400 51820 6 knapp "This is the craziest \
object that I have ever seen in my life. Someone \
should really use a few nights of Keck time and see."
David Schlegel,
schlegel@astro.princeton.edu