VaST [Variability Search Toolkit]A software for variable star detection
| ||||||||||
For a detailed description of the code, please see the VaST paper. The latest version of the code is available on GitHub.
VaST is a software tool for finding variable objects on a series of astronomical images. The images (CCD frames or digitized photographic plates) must be taken with the same instrument using the same filter and saved in the FITS format. CCD images should be calibrated (dark-subtracted and flat-fielded). The input images may be shifted and rotated with respect to each other, but they have to have the same scale (arcsec/pix). The images in a series should overlap by at least ~40% to ensure successful cross-identification. VaST performs object detection and aperture photometry using SExtractor on each image, cross-matches lists of detected stars, performs magnitude calibration with respect to the first (reference) image and constructs a lightcurve for each object. The sigma-magnitude, Stetson's L variability index, Robust Median Statistic (RoMS) and other plots may be used to visually identify variable star candidates. If data permit, period-search techniques may be employed to identify periodic variable stars among the candidates. All lightcurves are constructed in arbitrary magnitudes with zero magnitude corresponding to the background level on the reference image. If there are reference stars with known magnitudes in the field of view, all magnitudes can be later converted to the standard system. Unlike software based on the image subtraction method, VaST can be used in case of unstable PSF (e.g., bad guiding or with digitized wide-field photographic images). VaST is not tied up to any external catalog and WCS (if operated in its main "variable star search" mode), so it can be used on images taken with a telephoto lens as well as with a 2.6-m class telescope. If needed, absolute astrometric calibration may be performed through an interface to the Astrometry.net code allowing one to perform automatic magnitude scale calibration and variable star identification (however, this will work only if the input images are large enough to perform blind plate solution). VaST is written in C (and partly in BASH scripting language) for GNU/Linux operating system. The it is also tested on macOS and FreeBSD. The best practical way to run VaST under Windows is through Linux installed in a virtual machine (e.g. VirtualBox, see also a collection of pre-built VirtualBox images). The AAVSO forum thread mentions attempts to run VaST under Windows with Cygwin, but the results seem to be mixed. I haven't tested the code with Windows Subsystem for Linux (WSL). In principle, VaST should be able to work with WSL, but one will need to set up a working X window system on Windows before attempting to run VaST interactively. VaST is successfully tested on Raspberry Pi 4 (controlled by Raspberry Pi OS aka Raspbian). On all the supported systems VaST is able to utilize multiple processing cores to perform most of its computations in parallel. VaST is free software: you can redistribute and/or modify it under the terms of the GNU General Public License. The program is in a continual, albeit slow, state of development. You are welcome to contribute bug reports, patches and feature requests: please write me at kirx[at]scan.sai.msu.ru ScreenshotsVarious VaST subroutines (lightcurve plotter, image viewer) may be started automatically from the main program or manually by a user from a terminal. Names of the corresponding executable files are given under each screenshot. Click on images to see them in high resolution:
Video (screencast) voice comments are in Russian only, sorry... I hope to prepare an English version for quite a few years already...
Getting VaSTTo compile and use VaST you'll need:
The best way to get VaST is to clone the current development version from GitHub: git clone https://github.com/kirxkirx/vast.git Alternatively, you may download the stable version of VaST from this page (~22M). You may also want to download the sample data set from here, 24M). The older versions of VaST may be found here. You may download VaST with the command wget -c ftp://scan.sai.msu.ru/pub/software/vast/vast-latest.tar.bz2
unpack the archive containing the program
tar -xvjf vast-latest.tar.bz2
and compile it by running
cd vast-1.0rc87
make
(or 'gmake' if you are not on Linux).
At this point VaST should be ready for work.
If compilation fails, read the output carefully. Most probably it can't find some necessary libraries, header files or external programs. Please install the required programs and try to compile VaST again. If you are sure that the needed libraries and headers are installed in the system but the compiler cannot find them, try to open Makefile with a text editor and change the libraries locations to match your environment. Very old versions of VaST require PGPLOT, CFITSIO and GSL libraries to be installed in your system, but most likely you don't want to use such an old VaST version. Special note for Ubuntu users! The following installation procedure is recommended:
sudo port install gcc12 wget xorg-libX11 libpng
or, alternatively, through Homebrew:
sudo port install gcc wget libx11 libpng
(note the difference in the X11 libraries package name).
VaST will not compile with clang, you do need to install gcc, gfortran, g++...
Previously, a common problem on macOS systems was the mismatch between the versions of a manually installed gfortran and the default system gcc. This should be resolved by now, but if the version mismatch affects you, you may try to manually install gcc, g++ and gfortran as described here add the directory with the new binaries in the beginning of your PATH, the command should look something like export PATH=/usr/local/gcc-6.3.0/bin:$PATH
Create a symbolic link to gcc in the directory with the compiler binaries:
cd /usr/local/gcc-6.3.0/bin ;
ln -s gcc gcc-6.3.0
go back to the VaST directory and compile VaST with
make
Generally, it would be the easiest for VaST if 'gcc' and 'gfortran' commands
first appearing in the PATH point to compatible versions of gcc and
gfortran. Note that you don't need to modify PATH permanently by editing
.bashrc file (or an equivalent) - it is sufficient to export the correct PATH
before running 'make'.
Special note for FreeBSD users: VaST scripts may fail to compile the internal copy of PGPLOT library. If that happens, please install PGPLOT system-wide with the command (as root): pkg install pgplot
then try to compile VaST again. In order to use VaST you will need the
following packages
pkg install bash gmake gcc curl wget libx11
If you want to do PSF-photometry with VaST, but have no root access to install ATLAS library using the system's package manager - there is a way to compile and install all the needed programs and libraries at your home directory, but it's a pain. Here is what worked for me on CentOS 8: # Download FFTW, compile and install it using ./configure --prefix=$HOME && make && make install # Download ATLAS, compile and install it using mkdir my_build_dir cd my_build_dir ../configure --cflags="-fPIC" --cripple-atlas-performance --shared --with-netlib-lapack-tarfile=$HOME/src/v3.11.0.tar.gz --prefix=$HOME # --cflags="-fPIC" and --shared --with-netlib-lapack-tarfile=$HOME/src/v3.11.0.tar.gz # may not be necessary as the dynamic libraries will fail to build anyway, # only the static libraries will be built. # Install whatever we have: make make install # Clone SExtractor repository from GitHub, then ./autogen.sh # Now manually edit configure script adding after the line # '$as_echo_n "checking for library containing clapack_dpotrf... " >&6; }' # the following new line ac_cv_search_clapack_dpotrf="$HOME/lib/liblapack.a $HOME/lib/libcblas.a $HOME/lib/libatlas.a" # then ./configure --prefix=$HOME --with-atlas-libdir=$HOME/lib --with-atlas-incdir=$HOME/include --with-fftw-libdir=$HOME/lib --with-fftw-incdir=$HOME/include make make install # Clone PSFEx repository from GitHub and follow the same installation # procedure as for SExtractor: ./autogen.sh # Now manually edit configure script adding after the line # '$as_echo_n "checking for library containing clapack_dpotrf... " >&6; }' # the following new line ac_cv_search_clapack_dpotrf="$HOME/lib/liblapack.a $HOME/lib/libcblas.a $HOME/lib/libatlas.a" # then ./configure --prefix=$HOME --with-atlas-libdir=$HOME/lib --with-atlas-incdir=$HOME/include --with-fftw-libdir=$HOME/lib --with-fftw-incdir=$HOME/include make make install # # I also have export LIBRARY_PATH=$HOME/lib:$LIBRARY_PATH export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH export C_INCLUDE_PATH=$HOME/include:$C_INCLUDE_PATH export CPLUS_INCLUDE_PATH=$C_INCLUDE_PATH:$CPLUS_INCLUDE_PATH export PATH=$HOME/bin:$PATH # in $HOME/.bashrc # # After compiling/installing SExtractor and PSFEx you'll be able to run VaST # in the PSF-photometry mode using '-P' flag like this: ./vast -u -P ../sample_data/f_72-0* Using VaSTBrief usage instructions in Russian may be found here.To try VaST, download the sample data set, unpack it
wget -c ftp://scan.sai.msu.ru/pub/software/vast/sample_data.tar.bz2
change to the VaST installation directory and run the program
tar -xvjf sample_data.tar.bz2
cd vast-1.0rc87
After a brief computation the variability search window will be opened. Click
on any star on the sigma-magnitude plot to inspect its lightcurve. Click on
any point on the lightcurve plot to see an image from which this point
comes. The star on the image will be marked with a red cross, a red circle
around the star corresponds to the aperture used to measure this image.
You may switch from sigma to other variability indexes by pressing 'M' or 'N'
on the keyboard in the sigma-magnitude plot window.
./vast ../sample_data/*fit If you quit the program without deleting any data, you can restart it without doing all the computations again by running:
./find_candidates aa
To calibrate the magnitude scale using comparison stars with known magnitudes within the field of view use this script:
util/magnitude_calibration.sh
It will ask you to specify one or more comparison stars and their magnitudes.
For each comparison star, click on this star on the displayed image then
enter its catalog magnitude in the terminal window.
After the comparison stars have been specified, close the image window with the right mouse click and inspect the relation between instrumental and catalog magnitudes. This relation may be fitted by a linear function with the slope fixed to 1, linear function with a free slope, second degree polynomial (parabola) or the Bacher et al. (2005, MNRAS, 362, 542) magnitude calibration relation a.k.a "photocurve". This option is suitable for processing digitized photographic plates. Use (P) key on your keyboard to change the fitting function. For most CCD data sets, linear function with the fixed slope provides a good fit. The data points may be weighted according to their estimated errors or weighted equally. Use (W) key to change weighting. Close the dialog with the right mouse click to apply the calibration to all stars when you are satisfied with the fit. If the image field of view is large enough to be blindly solved with Astrometry.net code (as described below), one can automatically match stars detected on the image to the UCAC4/APASS catalog by running the magnitude calibration script with a command line argument specifying the observing band:
util/magnitude_calibration.sh V
The acceptable bands are:
"C" - UCAC5 unfiltered red band,
"B" - APASS B band,
"V" - APASS V band,
"R" - APASS R band,
"r" - APASS r band,
"i" - APASS i band. Magnitude calibration against APASS can be performed
only if the field of view overlaps with the APASS survey footprint.
Note that the automated magnitude calibration will not work with the
sample data set due to its small field of view.
In order to save the current work, use the script
util/save.sh my_favorite_field_name
If the field name was not provided, the script will ask for it. To restore previous
work, use
util/load.sh my_favorite_field_name
or just copy all saved files from directory [my_favorite_field_name] back to the VaST directory.To delete all files from the previous work, you can use:
util/clean_data
There is a tool to automatically identify variable stars detected with VaST. It requires a working Internet connection at runtime. The tool may or may not work for your field depending on luck. If the field of view is greater than, say, 30' and the image quality is fine - the odds that the automatic plate solution will work for your field are pretty good. Here is how to use it... To identify the star number 00190 in the current dataset use the following command:
util/identify.sh out00190.dat 25
where "out00190.dat" is the lightcurve file of the star and "25" is the
estimated field of view size in arcminutes. It is wise to set this value a little bit smaller
than your best-guess field of view. If no field of value is given, the
default value of 40' will be assumed. Note, that the field of view
identification may take really long time (up to ~30 minutes), especially for small
fields. Please, do not interrupt the process - just wait for a result. If
the field identification failed, try to restart it with smaller estimated
field of view size. On successful identification, the script will print the
equatorial coordinates (J2000) of the star and will attempt to query popular
databases to check if this star is already known as a variable. Note, that
the coordinates returned by the identification script may be up to 10" off.
We strongly advise to use the Aladin
interactive sky atlas to verify the identification and obtain more
precise star coordinates from some astrometric catalog (e.g., USNO-B1.0).
The automatic identification function relies on tools provided by Astrometry.net. For more details visit
http://astrometry.net/. Note, that
according to the
use conditions of the index files required to run the Astrometry.net
code, you need to be ready to share the images solved using this software
with any other interested party including the Astrometry.net team. According
to the same use
conditions: "If the indexes are used in any scholarly work,
research grant proposals or other scientific or academic
publications, the Astrometry.net project must be properly cited
and acknowledged. The paper to cite is the Lang et al paper on our
bibliography page http://astrometry.net/biblio.html.
The acknowledgment text is "This research made use of tools provided
by Astrometry.net." The VaST software developers assume, that this statement
concerns the work which uses "util/identify.sh" script from the VaST
package.
A note on the graphical user interfaceThe VaST GUI is based on the PGPLOT library. The main reasons are that this library is perfectly suited for displaying and editing data and image plots and is so easy to use from the developer's standpoint. The problem is that the resulting interface may look counterintuitive. The unusual thing about the VaST interface is that it has no buttons. (There is a way to program a GUI with buttons and graphical input fields in PGPLOT, but, trust me, you'll not be impressed by its look and usability.) So, whenever a user has a choice between multiple possible actions, instead of clicking a button to execute the desired action, a user will have to physically press a key on a keyboard. The list of possible keys is always displayed by VaST at the terminal. Please have a look there to know your options. Some actions are common to all VaST windows:
Using VaST with scanned photographic platesIf you have to deal with digitized photographic materials, you'll want first to convert TIFF images (which are typically produced by scanner software) to the FITS format. It can be easily done with the tiff2fits converter. Do not forget to change your images to positive (white stars on black sky). For example:
./tiff2fits -i input.tiff output.fits
Also, you'll have to enter the information about the date and time of the
observation in fits header. This can be done, among other ways, using the
modhead program from the CFITSIO
examples page, MissFITS or edhead from the WCSTools package. You may
enter the observation date and time information in the image header by generating
the usual "DATE-OBS" and "EXPTIME" header keys, or by simply putting the
middle-of-exposure Julian Date into the "JD" key.
You will most likely need to modify the default SExtractor settings stored
in the "default.sex" file located in the VaST directory. Please use
"default.sex.beta_Cas_photoplates" and "default.sex.PHOTO" as working examples
(these files are also found in the VaST directory).
Say, you have digitized photographic images in "photo_data" directory. To run VaST on them, change to the VaST directory and run the program
cd vast-1.0rc87
The "-o" parameter tells VaST to use formulas (1) and (3) from
Bacher et al. (2005, MNRAS, 362,
542) a.k.a. "photocurve" for magnitude calibration. This technique provide slightly better
results for the aperture photometry on photographic data compared to the
second-order polynomial used in VaST by default for magnitude calibration.
However, the standard calibration technique should also provide reasonably
good results (see Figure 2 in this paper).
The "-j" parameter that forces VaST to use a linear position-dependent
magnitude correction (i.e. stars on one side of the image being
systematically fainter than on the other side) might also be useful for
photographic plates photometry.
./vast -o ../photo_data/*fit An example photographic dataset is available here. Beware, it's about 1G in size! You may find some more details in the ArXiv:1403.5334 e-print "A search for new variable stars using digitized Moscow collection plates". Search for transients with VaSTApart from the standard "variable star search mode", where a long series of images is processed and lightcurves of detected stars are explored using the sigma-magnitude plot, VaST can be also used in the specialized "transient detection mode". In this mode VaST will process a series of four images: two first-epoch (reference) images and two second-epoch images. VaST will generate an HTML report containing a list of candidate transients. The transients are defined as either objects that were not visible on the reference images, or where at least 1 mag. fainter compared to the second-epoch images. Two first-epoch images are needed because image artifacts and star blending may cause VaST to miss a star on a single reference image, so it will appear as a false transient search mode is available. If there are two reference images - chances that the program will fail to detect a star at both of them is greatly reduced. Also note that two different reference images are needed: it would be impossible to "cheat" the program by feeding the same image to it twice. An example dataset that allows one to test the transient search mode is available here. The unpacked archive will contain two sub-directories with reference and second-epoch images. The search should be conducted in two steps:
# Process images with VaST
./vast -x99 -ukf ../transient_detection_test_Ceres/reference_images/* ../transient_detection_test_Ceres/second_epoch_images/*
# Run script that will generate an HTML report
util/transients/search_for_transients_single_field.sh
Be prepared - the script will ask you to download a few catalogs needed for its work, including Tycho-2, the catalogs are more about 200M in size. The transient search report will be placed in transient_report/index.html that should be opened with your
favorite web browser. The report will contain a few false candidates and some good ones including asteroids and variable stars.
Known limitations:
Some switches that you may use with VaST"-P" — perform PSF-fitting photometry with SExtractor and PSFEx."-9" — use DS9 instead of VaST's own viewer pgfv to view FITS files. "-u" — always assume UTC time system, don't perform conversion to TT. "-k" — force VaST to ignore the "JD" keyword in FITS headers. "-x2" — accept stars with non-zero SExtractor flag: 2 - blended stars, 4 - saturated stars, 99 - accept everything. "-e" — failsafe mode: only stars detected on the reference frame will be processed. "-p" — tell VaST that it deals with a linear detector. This flag is useful for well calibrated CCD images, but for some CCD images it is better not to use it. Test what works best for your particular dataset! "-o" — tell VaST that it deals with a digitized photographic plate. The magnitude calibration between frames will be done using the relation proposed by Bacher et al. (2005, MNRAS, 362, 542). "-l" — perform sigma-filtering of lightcurves. "-r" — assume that images are not rotated with respect to each other by more than 3 degrees. To get the full list of the available command line arguments run:
./vast -h
Description of output files produced by VaSTAfter processing an image series, VaST will create a number of files in its working directory that may be useful for a future work or problem debugging.
VaST mailing listIf you would like to receive updates about new versions of VaST, please enter your e-mail address in the form below This mailing list is really low-traffic.Release notes12 Dec 2023: vast-1.0rc87: * Changed the deprecated VizieR vizier.u-strasbg.fr domain to vizier.cds.unistra.fr VaST will now always use the CDS VizieR, and will not try to use mirrors as they fail often. * When determining CCD gain in e-/ADU from a FITS header, prefer EGAIN over GAIN keyword. * New script 'util/constellation.sh' prints the constellation name for a given J2000 position. This is a wrapper around WCSTools + Patrick Glaschke's code described in https://arxiv.org/abs/1008.3966 * New script 'util/planets.sh' prints geocentric positions of major planets for a given JD(UT). The positions are retrieved from JPL HORIZONS https://ssd.jpl.nasa.gov/horizons/ * Similarly to 'util/planets.sh', the new script 'util/moons.sh' prints geocentric positions of selected (bright and distant) planetary moons. * Similarly to 'util/planets.sh', the new script 'util/comets.sh' prints geocentric positions of bright comets (listed at http://astro.vanbuitenen.nl/comets ) for a given JD(UT). The specified JD(UT) should be near-current for the list of bright comets to be relevant! The comet positions are retrieved from JPL HORIZONS https://ssd.jpl.nasa.gov/horizons/ * Multiple improvements in the NMW transient search script 'util/transients/transient_factory_test31.sh' * Do not perform slow VizieR query for a photometric catalog when identifying a star using 'util/identify.sh' or 'U' in the lightcurve viewer. * 'vast_summary.log' now includes an estimate of the limiting magnitude of the reference frame. The magnitude is in the instrumental units (negative value) and gets updated to actual magnitudes as soon as magnitude calibration is performed with 'util/magnitude_calibration.sh'. * VaST can now handle TICA TESS Full-Frame Images that may be found at https://archive.stsci.edu/hlsp/tica#section-c34b9669-b0be-40b2-853e-a59997d1b7c5 * The 'pgfv' fits viewer will now exit with a non-zero exit code if 'Q' is pressed on the keyboard. This may be helpful for manually weeding out bad images with a loop like this 'for i in *fts ;do ~/current_work/vast/pgfv $i && continue ; mv $i bad/ ;done' pressing 'Q' for bad images and 'X' or right-click for good ones. * The new script 'lib/astrometry/strip_wcs_keywords' will strip all WCS-related keywords from the header of an input FITS image. This may be useful to ensure that there is no mixup between an old WCS solution that may have been present in the image and a new one produced with Astrometry.net 07 Apr 2023: vast-1.0rc86: * Experimental support for TESS Full-Frame Images (FFIs). In terms of photometric precision, VaST currently cannot compete with the analysis codes specifically designed for TESS. It is rather an attempt to apply quick mass-photometry to TESS images to enable discovery and characterization of bright and high-amplitude (by TESS standards) variable stars. TESS has 4 cameras and each camera has 4 CCD chips. VaST can process a series of images taken with one chip of a given camera during one TESS observing sector at a time. The command line for processing a set of TESS FFIs with VaST may look like this: './vast -u -f -t2 --selectbestaperture --autoselectrefimage --sysrem 3 /data1/TESS/sector_24/tess*-4-4-*' where '-u' explicitly sets the time scale to UTC, '-f' disables the graphical output, '-t2' set the photometric calibration type to 'zero-point offset only' (found most appropriate for TESS FFIs), '--selectbestaperture' for each object VaST will select aperture size that minimizes the lightcurve scatter, '--autoselectrefimage' let VaST automatically select the best reference image, '--sysrem 3' apply 3 iterations of SysRem to suppress residual systematics. As much as 9 SysRem iterations may be appropriate for TESS FFIs. The numbers '-4-4-' in the files names are the camera and CCD chip numbers. Processing TESS FFIs with VaST requires considerable computational resources. You may try to make cutouts (with 'util/fitscopy' or other tools) rather than processing full-chip images. You may want to process only the FFIs with good data quality flags ('DQUALITY= 0' key in FITS header). TESS FFIs may be downloaded at https://archive.stsci.edu/tess/bulk_downloads/bulk_downloads_ffi-tp-lc-dv.html * New experimental moving object photometry mode: './vast --movingobject ../moving_object_images/*.fit' VaST will ask you to mark moving object position on each image with a click. The intended use is comet/asteroid/space junk photometry. Only one moving object can be measured on a series of images at at a time. The lightcurve filename of the moving object is listed in vast_summary.log * Automatic magnitude calibration scripts now support g magnitudes derived from APASS or PanSTARRS1 catalogs. * The period search routine 'lib/ls_compute_periodogram' can now compute the Lomb-Scargle periodogram and the associated False Alarm Probabilities as defined by https://ui.adsabs.harvard.edu/abs/1982ApJ...263..835S/abstract https://ui.adsabs.harvard.edu/abs/2018ApJS..236...16V/abstract https://ui.adsabs.harvard.edu/abs/1992nrca.book.....P/abstract (the "Numerical recipes in C" book). As these False Alarm Probabilities have limited practical use (as they assume independent Gaussian noise in measurements), Lomb-Scargle periodogram has virtually no advantages over Deeming's Discrete Fourier Transform. But co-authors and referees keep asking for is as Lomb-Scargle is so popular, so here is an implementation. As with the DFT and Lafler & Kinman (string-length) period search implementations in VaST, the Lomb-Scargle periodogram is implemented as a command line tool (the periodogram file it produces 'ls.periodogram' may be visualized with 'gnuplot'). The Lomb-Scargle support in the interactive online period search tool integrated with VaST is coming soon. * VaST will not try to update the leap second 'tai-utc.dat' file as there is no more an official way to do it: the links http://maia.usno.navy.mil/ser7/tai-utc.dat and ftp://toshi.nofs.navy.mil/ser7/tai-utc.dat are down forever. If you know how to get updated versions of 'tai-utc.dat' in the future - please let me know! * VaST will try to discard images with elongated stars resulting from bad tracking. This can be disabled by specifying the new command line option '-l' or '--nodiscardell'. This is disabled automatically if '--photocurve' magnitude calibration is used as photographic plates often suffer from bad guiding and low number of images (so we cannot afford discarding a few suspicious ones.) The line 'Number of identified bad images:' in 'vast_summary.log' now shows a sum of bad images identified via the 'elongated stars' and 'lightcurve outlier' routes. * 'util/get_image_date' now understands input dates without quotation marks 'util/get_image_date 2020 10 27 18:00' and as a fraction of the day 'util/get_image_date 2020 10 27.75' * 'lib/deg2hms_uas' converts the input in decimal degrees to sexagesimal degrees:arcminutes:arcseconds with the 10 microarcsecond accuracy of the output. Example usage: 'lib/deg2hms_uas 126.59917135396 -50.96207264973' 'lib/deg2hms' prints out the output with the usual 0.1 arcsec accuracy. * Removed the internal copy of sextractor-2.25.0, keeping 2.25.2 and 2.19.5 * Retired the old VaST implementation of SysRem 'util/sysrem' as it started causing problems with cod maintenance. The new SysRem implementation 'util/sysrem2' is now the only one and 'util/sysrem' is a symlink to 'util/sysrem2' kept for backward compatibility. * VaST can now be used from behind an HTTP/HTTPS proxy: just export 'http_proxy' and 'https_proxy' variables before running VaST. For example: export http_proxy=http://squid.sao.ru:8080 export https_proxy=http://squid.sao.ru:8080 where squid.sao.ru:8080 is the name and port of the proxy server. * 'util/phase_lc' is a new tool to fold a lightcurve with given light elements: util/phase_lc out01234.dat 2459088.0123 0.567891 > phased_lightcurve_01234.txt The phased lightcurve can be plotted with gnuplot or some other plotting tool. The similar tool 'util/phase_and_bin_lc' can also bin the lightcurve in phase. * The new tool 'util/bin_lightcurve_in_time' can bin the input lightcurve in time. Example use: 'util/bin_lightcurve_in_time out01234.dat 120 > binned_lightcurve_120sec.dat' will bin the lightcurve file out01234.dat in 120s time bins and write the binned data to 'binned_lightcurve_120sec.dat'. * The new tool 'util/imstat_vast image.fits' will print out some basic image statistics. Works pretty much as 'util/colstat', but for an image instead of a data column. The intended use is automating image quality control. The tool 'util/imstat_vast_fast' is the faster version of 'util/imstat_vast' that outputs fewer parameters, but works ten times faster (by avoiding sorting the image array). * Added robust linear fit option to the absolute magnitude scale calibration interface. (It is now the new default calibration function, as usual, you may change the calibration function by pressing 'P' on the keyboard.) The recommended way to automatically calibrate the magnitude scale is now: 'util/magnitude_calibration.sh V robust_linear' You may also try to fit zero-point offset only: 'util/magnitude_calibration.sh V zero_point' Here V is the filter name, the supported filers are BVRIri As usual 'util/magnitude_calibration.sh V' will work only if the image can be automatically plate-solved. If the image is too narrow-field, specify the comparison star magnitudes manually by running 'util/magnitude_calibration.sh' without command line arguments. * VaST can now handle RGB DSLR images as input. It performs on-the-fly conversion extracting green channel and discarding red and blue channels. The conversion is equivalent to: 'util/fitscopy rgb_image.fit[*,*,2:2] g_image.fit' Note that the converted images are temporarily stored in 'converted_images/' subfolder of the vast working directory meaning you have to have enough disk space to copy the input set of images. An external program like 'siril' ( https://siril.org/ ) can be used to convert RAW DSLR images to FITS format (that can be read by VaST) and stack the images if needed. * In single-image inspection mode ('./sextract_single_image image.fits') you may now calibrate the magnitude scale automatically by pressing '4' on the keyboard. This will work only if the image can be automatically plate-solved. You'll need to enter the filter name in the terminal. * If manual (keyboard key '2') or automatic (keyboard key '4') magnitude calibration is performed in single-image inspection mode ('./sextract_single_image image.fits'), the calibrated magnitudes of all the detected stars are saved to 'image00000.cat.calibrated'. * The new 'vast' command line option '-K' or '--nodateobskeyword' tells the code to ignore DATE-OBS key and instead try to determine the observation time from other keywords. This (together with '-k' or '--nojdkeyword' option to ignore 'JD' keyword) is implemented only in './vast', but not in './pgfv' or 'util/get_image_date' * VaST will now use SHUTOPEN instead of DATE-OBS to read the observation start time if both DATE-OBS and SHUTOPEN are present in the FITS image header. * VaST can now get the middle of exposure time from MJD-OBS keyword. * Fixed segfault with '--type' photometric calibration type argument (the short version '-t' was working fine). * Relaxed the way for constant stars (listed in vast_list_of_likely_constant_stars.log) are selected as this affects automatic magnitude calibration. There may be only a small overlap between 1000 brightest stars in the field used for catalog search and the list of non-variable stars. * VaST scripts now write to '>&2' instead of '/dev/stderr' so they should be able to run fine under 'su'. * Fixed the bug with the internal copy of SExtractor not compiling on Ubuntu 20.10 (new GCC supplied with Ubuntu requires '-fno-common' flag to compile SExtractor). * VaST will warn a user if the input is a compressed FITS image. The compressed images should be unpacked with 'util/funpack' before processing. * Added the new VaST test script 'util/examples/git_pull_run_test_email_on_failure.sh' that can be run from a cron job and e-mail a test report. * Added CFITSIO example program 'util/imarith' allowing simple arithmetic operations on images (used in test script). * You may now run the sigma-clip filter on an individual lightcurve file: lib/new_lightcurve_sigma_filter 3.0 out12345.dat # here 3.0 is the sigma-clip level * VaST is now able to read the CCD gain value from CVF keyword written by N.I.N.A. camera control software. * 'lib/deeming_compute_periodogram' will now write a binned power spectrum and spectral window to 'binned_powerspectrum.periodogram' file that can be displayed with gnuplot or other tools. 20 Oct 2020: vast-1.0rc85: * 'util/get_image_date' can be used to convert JD to calendar date/time: util/get_image_date 2456909.72911 it will understand input as MJD (distinguishing it from JD based on the numerical value): util/get_image_date 58020.39 Remember the 0.5 day difference between MJD and truncated JD! Never use truncated JD, in my opinion MJD should also be avoided. Spell out a full JD to specify time for the sake of avoiding confusion. Computers nowadays can handle the two extra digits, they really do. 'util/get_image_date' can also be used to convert calendar date/time to JD: util/get_image_date 2014-09-09T05:29:55 it will also understand util/get_image_date '2014-09-09 05:29' * The new script 'util/fov_of_wcs_calibrated_image.sh' will print the field of view, image scale and the image center position of a WCS-calibrated FITS image. * The new tool 'util/fits2png' will produce a PNG finder chart with no labels from the input FITS image. Use 'util/make_finding_chart' to produce a finder chart with labels. In all cases the output image will be written to 'pgplot.png' file in the current directory. * Lightcurve files (out*.dat) listed as automatically selected candidate variables in vast_autocandidates.log as well as the ones listed by user as previously known variables in vast_list_of_previously_known_variables.log are now highlighted by the FITS viewer './sextract_single_image' * You may now specify on the command line the directory that contains one level of sub-directories with images: './vast -u -f ../vast_test_ASASSN-19cq' where the directory ../vast_test_ASASSN-19cq contains sub-directories '2019_05_15 2019_05_25 2019_05_26 2019_06_03 2019_06_10 2019_06_24' with FITS images. Note that VaST will only go two levels down the directory tree, i.e. images from the sub-directory '2019_05_15/bad_images' in the above example will not be processed. * VaST will now understand the exposure start time if it is written in 'SHUTOPEN' keyword instead of 'DATE-OBS'. This is to process the ZTF 'sciimg' images (see 'ZTF Image Access' at https://irsa.ipac.caltech.edu/Missions/ztf.html ). Example of running VaST on ZTF images: ./vast --UTC --poly /mnt/usb/ZTF_images_test_2/ztf_20180*_zg_*_q1_sciimg.fits * For star identification VaST will blindly trust the astrometric solutions produced with SCAMP ( https://www.astromatic.net/software/scamp ). In this case, VaST will not attempt to re-solve the image using a local or remote copy of the Astrometry.net code. The same is true if the FITS header indicates that the image was already plate-solved with Astrometry.net * When solving plates with a local copy of the Astrometry.net code, VaST will now try to refine the plate solution by performing the second run of the Astrometry.net code restricting the search space by using the image center and scale derived from the first run. * Updated the flag/weight image generation rules for better handling stacked images with black areas around the image edges. * The file names in the vast_magnitude_calibration_details_log directory now include image number as well as the image name: useful if multiple input images from different directories have the same name. * VaST should warn the user if the reference image seems to be of poor quality (has much less stars) than other images in the series. If this happens, you may want to manually specify another reference image by putting it before all other images in VaST command line arguments. For example the command: ./vast ../images/image-012.fit ../images/image-*.fit will make VaST use 'image-012.fit' as the reference image. * Tweaks to the star list matching algorithm should allow VaST to process images with very few stars (less than 10). You may need to think twice about the magnitude calibration procedure for such images: the linear calibration ('--poly' option), or the zero-point-offset-only ('-t 2' option), or manually selecting the reference stars using 'diffphot' may be good options. * VaST will now try to parse the EXPOSURE keyword comment to determine if the exposure time is specified in minutes (like in DSS FITS images) or hours rather than seconds (most FITS images). * '--nomagsizefilter' option now switches off also the psf fit quality vs magnitude filter. If you don't find your target star in the PSF photometry mode, but it is there when the aperture photometry is used - that option might help. * Changed the maximum number of stars for plate solution in 'util/solve_plate_with_UCAC5' to 1000 in order to speed up communication with VizieR. You may change this default value by editing the line '#define MAX_STARS_IN_VIZQUERY 1000' in 'src/solve_plate_with_UCAC5.c' and recompiling VaST with 'make'. * The lightcurve plotter './lc' can fit a linear trend to the lightcurve (when viewing a lightcurve press '1' on the keyboard). The fitted line slope is now displayed in mag/day as well as the time of decline by two (t2) and three (t3) magnitudes for convenience. * The list of input images may be specified in the text file 'vast_list_of_input_images_with_time_corrections.txt' instead of the command line. The file should contain a path to one image per line. No white spaces should be present in the path (if there are white spaces - rename the files or create symlinks). A time correction in seconds or the Julian Date of the observation may be specified as the second column in the input file list. This correction will be applied to the image date derived from the FITS image header. If the Julian Date is specified - it will override the date derived from the FITS header. See the examples in 'vast_list_of_input_images_with_time_corrections.txt_example' and 'vast_list_of_input_images_with_time_corrections.txt_test'. * Fixed the "Produce list of minor planets with online MPChecker" function in the transient search mode. * To speed-up the plate solution step needed for star identification, VaST can now use a local copy of the UCAC5 catalog. The catalog files (5,3G) can be installed using the script 'lib/install_local_copy_of_UCAC5.sh' or they can be manually download from VizieR and placed in 'lib/catalogs/ucac5/'. If the files are not found, VaST will access the catalog online through VizieR, as before. * Added "A first catalog of variable stars measured by the Asteroid Terrestrial-impact Last Alert System (ATLAS)" Heinze et al. (2018, AJ, 156, 241) https://ui.adsabs.harvard.edu/abs/2018AJ....156..241H to the variable star candidate identification script 'util/search_databases_with_vizquery.sh' The catalog is accessed via VizieR: http://cdsarc.u-strasbg.fr/viz-bin/cat/J/AJ/156/241 * Changed the algorithm for automatic reference image selection (activate with '--autoselectrefimage' option). Now the reference image will be the image with the best seeing that doesn't have too few or too many stars compared to other images in the series. If the option '--autoselectrefimage' is not specified, the reference image will be the first image supplied on the command line, as before. * The script 'util/load.sh' used to restore a set of lightcurves, configuration and log files previously saved with 'util/save.sh' will now try to use 'find' command to copy the files. This is ten times faster than the old 'for' loop, but it will not ask if you want to keep the old configuration files (most importantly default.sex) and will just silently overwrite them, so please beware! * Disabled photometric error rescaling as it works imperfectly and causes lots of confusion. You may still perform the error rescaling manually by running 'util/rescale_photometric_errors' * The new script 'util/make_finding_chart_script.sh' will use Swarp ( https://www.astromatic.net/software/swarp ) to resample the input WCS-calibrated image to the North-up, East-left orientation and make a good-looking finder chart for the object specified by its RA and Dec. * You may display a WCS-calibrated image marking a specified celestial position on it by running: ./pgfv wcs_Sgr9_2020-9-1_17-35-4_002.fts -- 18:19:53.946 -30:41:19.78 here the wcs_*.fts file is the image, 18:19:53.946 -30:41:19.78 are the J2000 coordinates and '--' marks the end of the command line options (needed for the program to correctly interpret the negative declination). * Updated TODO list of desired features. * Limited the number of OpenMP threads to no more than 48. Tests on a machine with 192 virtual cores (2 CPUs x 48 cores x hyper threading) revealed that while it is beneficial to use all cores for running instances of SExtractor, all the internal VaST tasks parallelized with OpenMP actually run much faster when the number of threads is limited. * Updated the internal copy of sextractor-2.25.2 that can be compiled with gcc10. 07 Mar 2019: vast-1.0rc84: * VaST is now on Github ( https://github.com/kirxkirx/vast ), pull requests are welcome! * The new script 'util/get_gaia_lc.sh' gets Gaia G, BP and RP lightcurves for a Gaia DR2 ID that should be specified as a command line argument. You may get Gaia ID of a source at a given sky position by running 'util/search_databases_with_vizquery.sh' The lightcurves are saved in the folder 'gaia_lightcurves/' within the VaST main directory. * 'util/colstat' will take a column of numbers from a standard input and compute mean, median, sigma and other values describing the column. For example, to compute statistics on the scatter of magnitude measurements among all stars that passed the selection, one may extract the second column from 'vast_lightcurve_statistics.log' using 'cat' and 'awk' and send it to 'util/colstat' like this: cat vast_lightcurve_statistics.log | awk '{print $2}' | util/colstat * The script 'util/search_databases_with_vizquery.sh' is trying to guess the star's spectral type from its 2MASS colors. The color-spectral type relation is updated to correspond to "A Modern Mean Dwarf Stellar Color and Effective Temperature Sequence" table by Eric Mamajek http://www.pas.rochester.edu/~emamajek/EEM_dwarf_UBVIJHK_colors_Teff.txt instead of the older reference http://adsabs.harvard.edu/abs/1988PASP..100.1134B * Upgraded the internal copy of SExtractor to version 2.25.0 while keeping 2.19.5 as a the automatic fall-back option. The new SExtractor cannot be compiled on very old systems and it requires autoconf that is missing on BSD systems and (by default) in Ubuntu. If a system-wide installation of SExtractor is found (by simply testing for the presence of 'sex' executable in $PATH), the internal copy of SExtractor is not compiled at all to speed-up compilation. * Added integration with wcs-addpv.py script ( https://github.com/evertrol/sippv ) as the second line of defense against the "TPV vs. SIP" problem: astrometry.net describes image distortions using the SIP convention while SExtractor/SWarp and other Astromatic tools expect the TPV convention. The script requires python+numpy+scipy+astropy to run. If you run astrometry.net code locally, the required python tools are already installed in your system. The old trick with VaST using 'xy2sky' from WCSTools to re-compute celestial coordinates in SExtractor catalogs still works, of course. Having the TPV distortions inserted in the images plate-solved with VaST is especially useful if you want to stack images with SWarp. 01 Oct 2018: vast-1.0rc83: * A careful review and refactoring of almost the entire VaST C-code tree was performed by Vladimir Bazilevich. * The variable identification script 'util/search_databases_with_vizquery.sh' will now try to identify the input object with Gaia DR2 in addition to USNO-B1.0 and display its Gaia DR2 variability status (VARIABLE/CONSTANT/NOT_AVAILABLE) and if it's listed in Gaia DR2 Cepheid, RR Lyrae, LPV or Short-timescale sources catalogs (see http://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/345&-to=2 ). You may test the new output by running: util/search_databases_with_vizquery.sh 22:02:43.29139 +42:16:39.9803 * The variable identification script will search the ASASSN-V catalog downloading its local copy (currently 71M) at first use. * VaST will now automatically update the local copies of ASASSN-V and VSX catalogs, as well as the asteroid database astorb.dat if they are more than one month old. The relevant code is in lib/update_offline_catalogs.sh You may run this script as 'lib/update_offline_catalogs.sh all' to download/update all catalogs. Note that the variable star catalogs are checked by the script util/search_databases_with_vizquery.sh in the following order: 1. VSX (offline copy) -- if available 2. ASASSN-V (offline copy) 3. GCVS (online) 4. VSX (online) 5. other catalogs in selected sky areas. The catalog search is performed until the first match is found. The (unintended) consequence of this is that the displayed variable star name may depend on the available catalogs. For example if the local copy of VSX is available, the search 'util/search_databases_with_vizquery.sh 34.8366337 -2.9776377' will result in "omi Cet" while if there is no local copy of VSX, the variable will first be found in the ASASSN-V catalog and its name will be displayed as "ASASSN-V J021920.94-025841.2". * VaST will now drop one of the top-ten brightest stars from the instrumental (image-to-image) magnitude calibration. The excluded star is the one that changes the instrumental magnitude/reference magnitude fit the most. This helps to handle the situation when the variable is one of the few brightest stars on frames. If the variability amplitude is sufficiently small to bypass the old outlier rejection algorithm, such variable could influence the magnitude calibration (due to its high weight as a bright star) and introduce artificial variability in other stars. Thanks to Vladimir Bazilevich for providing data that highlighted this problem! * Updated the dark frame subtraction ('util/ccd/ms') and flat-field division ('util/ccd/md') routines to better handle images with overscan. Numerous bug fixes in these routines. * The PSF-quality filter (active in the PSF photometry mode) is disabled for the brightest stars on frame (along with the A_IMAGE and FWHM_IMAGE vs. mag filters). The relevant code is in src/filter_MagSize.c * VaST was not recognizing the long version of the '--aperture' command line option. This has been fixed. * Updated the internal copy of CFITSIO to version 3.440. The HTTPS support is disabled in the library to avoid the dependence on libcurl - no web access is performed through the CFITSIO functions in VaST. 16 Apr 2018: vast-1.0rc82: * Fixed the bug that caused VaST to segfault in some circumstances. * Fixed the bug in the lightcurve viewer './lc' that was causing a crash when the input lightcurve file was outside the VaST working directory. * Fixed the broken counter of stars rejected as having a bad PSF-fit in the PSF photometry mode. The counter was stuck displaying the value for the reference image, not the current image. * Fixed the bug that caused the UCAC plate solver to work incorrectly on BSD systems. * Instead of relying on UCAC4 for both astrometry and photometry, the plate solver now relies on UCAC5 for astrometry and APASS (now accessed separately form the astrometric catalog) for photometry. If the field is not covered by APASS, the solver tries to use Pan-STARRS1 as the fall-back option. In this case, Pan-STARRS1 magnitudes are converted to APASS bands following Tonry et al. (2012, ApJ, 750, 99) http://adsabs.harvard.edu/abs/2012ApJ...750...99T The new solver name is 'util/solve_plate_with_UCAC5', the 'util/solve_plate_with_UCAC4' is now a symbolic link to the new plate solver kept for backward compatibility with old scripts. * Normally, VaST will use for the instrumental magnitude (image-to-image) calibration only the stars that were marked by SExtractor as non-blended. This causes problems if the majority of stars on an image are marked as blended. To handle this, VaST will now accept for magnitude scale calibration blended stars if less than 50% of stars matched between the current and reference images are marked as non-blended on both images. * The new command line option '--excluderefimage' will exclude the reference image from all the output lightcurves. This is useful if you use a deep stacked image as the reference for a series of individual-exposure images. With some additional scripting, this may also be used to incrementally add new observations as new images arrive, avoiding the need to reprocess all the available images of the field. Even if excluded, the reference image is still used as the photometric reference for the instrumental (image-to-image) magnitude scale. * The FITS image viewer './pgfv' now displays celestial coordinates of pixels if the image has the WCS information in its header. The pixel to celestial coordinates transformation is performed using 'xy2sky' from WCSTools. * Changed the lightcurve clipping algorithm used to compute the clipped standard deviation (the default variability statistic displayed by './find_candidates') to exactly match the description in Appendix A of Pashchenko et al. (2018, MNRAS, 475, 2326) http://adsabs.harvard.edu/abs/2018MNRAS.475.2326P * Revised the flag/weight image construction algorithm. * VaST is trying to identify candidate variables by applying magnitude-dependent cuts in multiple variability indices. Some details of this process may be found in the new log file vast_autocandidates_details.log which lists for each star the variability tests that it passed (if any) as well as conditions (prefixed with '-') that will cause the star not to be flagged as variable even if it passed one of the variability tests. Currently these conditions include the star being one of the few brightest or faintest stars on the frame or having a small reduced chi^2 value (large errors compared to the variability amplitude). The code relevant to the variability detection tests may be found in src/index_vs_mag.c and the variability detection thresholds for each test are set in src/vast_limits.h * The routine 'util/get_image_date' can now be used not only to get the observing time information from a FITS image header, but also to convert the calendar date to JD on the command line using the following syntax: util/get_image_date '2014-09-09T05:29:55' * Disabled OpenMP on systems with less than 1GB of physical memory as it was found to cause problems in low-memory conditions. * Updated the internal copy of cdsclient to version 3.84. * Updated the internal copy of GSL to version 2.4. 28 Dec 2017: vast-1.0rc81: * Implemented a new way of filtering-out blended sources. For each source VaST counts how many times it was detected and how many times it did not pass the magnitude-size filter. Sources that pass the filter less than a specified fraction of times are rejected. The minimum fraction of detections that should pass the magnitude-size filter is set by MIN_FRACTION_OF_GOOD_MEASUREMENTS in src/vast_limits.h (the default is 0.7) This helps to reject close pairs of stars that can be separated only on images with the best seeing, but still result in corrupted photometry. * Introduced a more efficient way of computing magnitude-dependent thresholds for variability indices. This should considerably speed-up computations for datasets containing >1000 sources. * For images that contain large blank areas with zero counts (like mosaics or images resampled to the north-up/east-left orientation) VaST will now construct and apply weight images. The weight images are needed for SExtractor to exclude the blank areas from background estimation. Incorrect background estimation may affect photometry of sources that are near the blank areas. The downside is that you cannot use WEIGHT_TYPE BACKGROUND together with the VaST-generated weight images. The previous versions of VaST were producing only flag images that were useful to flag false source detections near the blank image areas, but did not affect the background estimation for sources that do not get flagged. * Added a command line option that disables bad image identification: '-6' or '--notremovebadimages'. * Added routines to handle time records in images from the Aristarchos telescope. * Because of the way broken-down time to JD conversion is implemented, VaST still cannot handle sub-second timing. Now if the exposure start time is specified to an accuracy of better than one second in the FITS header, VaST will properly round it to the nearest second instead of truncating at the decimal point (taking the floor of the floating point number of seconds). * The new utility 'util/fix_image_date' may update FITS image header in order to specify the observing time in the "standard" format described by the two keywords: DATE-OBS (exposure start time in UTC) and EXPTIME (exposure duration in seconds). Note that this will work only if the exposure time is already written in the FITS header (or in the vast_list_of_input_images_with_time_corrections.txt file) in a format understandable to VaST. This utility may be useful to make the observing time information readable to software other than VaST. * The middle of exposure may now be specified with JDMID keyword in the image header. The old way of specifying the middle of exposure with JD keyword works too. * The manual magnitude calibration display now doesn't allow one to set magnitudes of saturated stars (identified by the SExtractor flag >=4). * VaST will not start if it fails to recognize a command line option. This should prevent typos in long option names from going unnoticed. * Access to online plate-solving and period search services is now working from inside a firewall that blocks ping. Originally, ping was (naively) used by VaST to test server availability. Now if ping fails the relevant scripts will try to send HTTP GET requests to the servers and conclude that VaST is running offline only if both ping and GET fail. * 'util/wcs_image_calibration.sh' will not re-run SExtractor if a catalog of sources corresponding to the given image was already created by VaST while processing the image series. This should speed-up the typical workflow: "process a series of images with VaST" -> "plate solve some or all of these images". * The external catalog matching routine 'util/solve_plate_with_UCAC4' iteratively improves plate solution by computing and applying local astrometric corrections. Each iteration involves the slow step of network communication with VizieR. If you are sure that the astrometric solution is good enough after the first iteration, you may set MAX_NUMBER_OF_ITERATIONS_FOR_UCAC4_MATCH to 1 in src/vast_limits.h to speed-up processing. 6 Oct 2017: vast-1.0rc80: * Added a workaround for the PV/SIP WCS convention problem that does not allow SExtractor to take into account the distortion corrections determined by Astrometry.net see http://adsabs.harvard.edu/abs/2012SPIE.8451E..1MS for a detailed discussion. The workaround is to use xy2sky routine from WCSTools to convert SExtractor-derived pixel coordinates to celestial coordinates rather than relying on SExtractor to do this conversion. * The magnitude measurements are now performed in multiple apertures for each image: ref. ap. = median(A_IMAGE) * CONST the reference aperture (the same used by previous version of VaST), here median(A_IMAGE) is the median semi-major axis lengths among all the sources detected at the current image, CONST = 6 (may be changed in src/vast_limits.h), ap. 1 = AP01 * ref. ap., by default AP01=-0.1 ap. 2 = AP02 * ref. ap., by default AP02=0.1 ap. 3 = AP03 * ref. ap., by default AP03=0.2 ap. 4 = AP04 * ref. ap., by default AP04=0.3 the values of AP* parameters may be change in src/vast_limits.h. The measurements in multiple apertures are used to reject blended sources if '--magsizefilter' is enabled (see below) or to select for each object an aperture producing its lightcurve with the smallest scatter. The second is an experimental option, it is disabled by default and may be activated by specifying '--selectbestaperture' or '-3' option on the command line. Please try and see if enabling this option actually improves the results with your data. * The magnitude-size filter now includes a battery of filters aimed at rejecting blended and extended sources. For each frame VaST rejects outliers on the instrumental magnitude vs parameter plots where the parameters are A_IMAGE, FWHM_IMAGE, mag(ref. ap.)-MAG_AUTO, mag(ref. ap.)-mag(ap. 1), mag(ref. ap.)-mag(ap. 2), mag(ref. ap.)-mag(ap. 3), mag(ref. ap.)-mag(ap. 4), A_IMAGE/B_IMAGE. The filtering is now enabled by default. You may disable it with '--nomagsizefilter' or '-2' command line options. If image contains many artifacts such as cosmic rays or hot pixels (often found in CMOS chips) that outnumber stars at some magnitude range, the filter may do more harm than good. If this is the case for your images, try disabling the filter and set more conservative SExtractor source detection parameters in default.sex Details about VaST magnitude-size filtering for each image may be visualized in gnuplot using the log files image*.cat.magsizefilter_* (for A_IMAGE) and image*.cat.magparameter*filter_* for the remaining filtering parameters. See SExtractor manual for the detailed description of A_IMAGE, FWHM_IMAGE, MAG_AUTO. * The magnitude calibration script now scales the photometric errorbars, not only the magnitudes themselves. This makes a big difference with 'photocurve' calibration at the bright end. * Photometric measurements with errorbars larger than MAX_MAG_ERROR=1.086/MIN_SNR=0.362mag (for the default MIN_SNR=3) are now ignored at all processing stages. You may change the maximum allowed error in src/vast_limits.h * VaST now tries to rescale the errorbars determined by SExtractor using the relation sigma_new=sqrt( (gamma*sigma_old)^2 + epsilon^2 ) where the coefficients gamma and epsilon are determined using all non-variable stars in the current dataset. This relation is inspired by the OGLE rescaling of photometric errors in DIA as described by http://adsabs.harvard.edu/abs/2017MNRAS.468.2189Z http://adsabs.harvard.edu/abs/2009MNRAS.397.1228W VaST is not allowed to make the errors smaller than the ones reported by SExtractor, so the parameters are forced to have the values gamma>=1.0 and epsilon^2>=0.0. Here epsilon may be interpreted as the systematic noise floor in the current dataset. The photometric errorbars rescaling is enabled by default. It may be disabled by running './vast --noerrorsrescale /path/to/images/*fit' or './vast -4 /path/to/images/*fit' The new line in vast_summary.log indicates if the errorbars rescaling was requested or not: Photometric errors rescaling: YES / NO The photometric errorbars rescaling is usually not needed in the PSF photometry mode, but should do no harm if applied. * VaST selects candidate variables (listed in 'vast_autocandidates.log' and highlighted in blue in './find_candidates') as objects that stand out in one or more variability index vs. magnitude plots. VaST also lists candidate non-variable stars in 'vast_list_of_likely_constant_stars.log' -- these are well-measured stars that do not stand out in multiple variability indexes simultaneously. The process of selecting candidate variables and candidate constant stars may now be controlled via the parameters set in 'src/vast_limits.h' (they follow the 'Automated selection of candidate variables' comment). See http://adsabs.harvard.edu/abs/2017MNRAS.464..274S for a detailed discussion of variability indexes implemented in VaST. * ./find_candidates now deletes the list of previously-viewed lightcurves kept in vast_viewed_lightcurves.log if started with no command line arguments ('recompute everything' mode). In earlier versions this file was kept which caused unexpected behaviour: every time the candidate viewer was run as './find_candidates aa' (in the 're-display previously-computed variability statistics' mode) it was highlighting all the previously-viewed objects, regardless of how many times variability statistics were recomputed with './find_candidates' * It is now possible to send a lightcurve to the online period search tool from the lightcurve viewer ('./lc') by pressing 'L' key when the lightcurve viewer was not started from the VaST directory. For example: cd /path/to/my_lightcurve_archive /home/path_to/vast-1.0rc80/lc my_lightcurve.txt * Implemented a more efficient procedure that should speed-up lightcurve writing on all systems supporting OpenMP. * Fixed the bug that prevented VaST from saving magnitude calibration details under some circumstances. The symptom was that vast_magnitude_calibration_details_log was an ASCII file instead of being a directory containing the calibration details. * VaST should now be able to understand observation start dates written in DATE-BEG FITS header keyword instead of the usual DATE-OBS (as recommended by the draft FITS Standard V4.0). * VaST may now open "slice of a data cube" i.e. an image with NAXIS = 3 and NAXIS3 = 1. Such slice number $CUBE_SLICE may be extracted from an actual 3D data cube named $INPUT_IMAGE_CUBE using 'fitscopy' example program from CFITSIO: fitscopy "$INPUT_IMAGE_CUBE[*,*,$CUBE_SLICE:$CUBE_SLICE]" \!"$OUTPUT_IMAGE_NAME" * VaST will now complain if it encounters a FITS image compressed by MaxIM DL. These images have to be uncompressed in MaxIM DL before they can be processed with VaST (or any other software that is not MaxIM DL - I love proprietary data formats). * Changed axis labels in './find_candidates' to indicate that the displayed values of MAD and IQR are scaled to sigma (to ease comparison with the mag-sigma plot). * The new '--starmatchraius' command line option may be used to tweak the star matching radius - the maximum distance in pixels (after applying XY coordinates transformation) between the objects detected on two images for these objects to be assumed the same. You may need to tweak this parameter in case you experience problems with blending. * Fixed the numerical overflow issue in the flat-field correction routine 'util/ccd/md' that was causing centers of saturated stars to appear black. * Added support for the I band to the magnitude calibration script 'util/magnitude_calibration.sh', as with the R band, I magnitudes are computed from APASS r and i magnitudes of the field stars using the color transformations by Jester et al. 2005 AJ, 130, 873 http://adsabs.harvard.edu/abs/2005AJ....130..873J * A safe value for one of the memory-management parameters MAX_MEASUREMENTS_IN_RAM in src/vast_limits.h is now set automatically at compile time based on the available amount of physical memory. * Minor bug fixes. 22 Jun 2017: vast-1.0rc79: * It is now possible to record values of any keywords from a FITS image header alongside the brightness measurements derived from this image. The keyword name and value are written in the lightcurve (out*.dat) files starting from Column 8. The list of keywords to be recorded is specified in the ASCII file vast_list_of_FITS_keywords_to_record_in_lightcurves.txt one keyword per line. You may use 'util/listhead file.fits' to list the available keywords. The intended use of this new keyword-recording feature is to enable external-parameter decorrelation of the brightness measurements (e.g. decorrelate the lightcurve with AIRMASS). As a by-product, it is now possible to keep free-format comments in lightcurve files after Column 7 that lists the image filename from which a given measurement was obtained. * If presented with a multi-extension FITS (like the one downloaded from the HLA) VaST will now extract sources only from the first image extension, assuming other image extensions in the file are data quality images (ignored by VaST). * If REMOVE_FLAG_IMAGES_TO_SAVE_SPACE is defined in src/vast_limits.h at compile-time, VaST will not keep the flag images it produces to filter-out detections along the image edges e.g. if overscan area is present on the image). This will save disk space. * Re-written the algorithm rejecting bad PSF fits in the PSF photometry mode ('-P' option). * A copy of NOVAS library updated to v3.1. * Updated the internal copy of CFITSIO library to v3.410. * Minor bug fixes. 29 Mar 2017: vast-1.0rc78: * If no image is specified for './sextract_single_image' on the command line - it will display the reference image. * VaST has a new experimental feature: it tries to suggest candidate variables. The code now produces two star lists: - vast_autocandidates.log - a list of candidate variables that stand out in mag-IQR or mag-1/eta plots. A cut on the reduced chi^2 value is also applied. The idea here is that photometric errors are usually underestimated, but not overestimated, so if variability is small compared to the estimated errors - it is likely not real. Keep in mind that the selected stars are just stars with high scatter or very smooth lightcurves, VaST doesn't know much beyond that. All candidate variables should be inspected by eye(!!!). It is especially important to inspect images of the candidate variables as they may reveal measurement problems like blending or bad CCD pixels that artificially increase the lightcurve scatter. - vast_list_of_likely_constant_stars.log - list of likely constant stars that do not stand out in mag-IQR, mag-1/eta and mag-weighted_sigma plots. The reduced chi^2 value is not considered here as the photometric errors may be highly underestimated (resulting in high reduced chi^2 values for constant stars). * A new experimental filter may be activated by running VaST with the option '-1' or '--magsizefilter': for each image it will try to filter-out extended or blended objects (that cannot be properly measured) by removing outliers on the size-magnitude plot. * When constructing lightcurves VaST now always ignores detections with SExtractor flags >7 as positions of such sources cannot be accurately measured anyhow. * VaST can now handle images that have few (about 50 or so) stars. If these are CCD images, it is highly recommended to use a linear function for magnitude calibration instead of the default parabolic one. To use linear magnitude calibration ru VaST with '-p' command line option: ./vast -p ../sample_data/*.fit * After creating lightcurves, VaST is trying to identify bad images in the input series as the ones which have a large percentage of outlier measurements associated with them. If measurements obtained from a given image are 3*sigma outliers in the lightcurves of at least 10% of objects detected on this image - probably something is wrong with this image and all measurements taken from it will be rejected. The first (reference) image never gets rejected. If the lightcurves are imported and not produced by VaST, you may run the bad-image-rejection procedure manually: lib/remove_bad_images 0.1 where 0.1 is the maximum allowed fraction of 3*sigma outliers in an image. * Now it is possible to set a non-zero MAG_ZEROPOINT in default.sex if you want. But generally speaking, it is not easy to know the exact value of MAG_ZEROPOINT for a reference image: you first need to calibrate it somehow and then run VaST with the fixed aperture size (like '-a5.0' for a fixed 5 pixel diameter aperture) because if you let VaST to choose an aperture on its own - the aperture may be inappropriate for the specified MAG_ZEROPOINT value. The standard practice is still to let VaST construct lightcurves in instrumental magnitudes (with MAG_ZEROPOINT 0.0) and then calibrate the magnitude scale using the script util/magnitude_calibration.sh * Improved thread creation logic for parallel image processing in SExtractor to avoid gaps when multiple processing cores are underutilized. This should lead to a slight increase in processing speed. * It is now possible to manually control the number of threads created by VaST by setting the environment variable OMP_NUM_THREADS before running VaST. Here is a BASH example of how to set the variable: export OMP_NUM_THREADS=5 (will set the number of threads to 5). This variable is respected even by parts of the code that do not rely on OpenMP and use the simple fork() to create multiple threads. You may also set N_FORK in src/vast_limits.h but this will affect only parts of the code parallelized with fork() (notably, parallel processing of images with SExtractor) but will not affect parts of the code parallelized with OpenMP. In that case, OpenMP will decide on the optimal number of threads for running the OpenMP-parallelized code at runtime depending on current system load. * Removed the default filter lib/remove_points_with_large_errors that was sigma-clipping the lightcurves based on estimated photometric errors. This filter may be slow and appears to be not very effective. You may run lib/remove_points_with_large_errors manually on your lightcurves and see if it helps or not. * Minor bug fixes. 24 Feb 2017: vast-1.0rc77: * Fixed the matching algorithm crash occurring if more sources are detected on the second image than on the first one (have no idea how this bug was hiding itself for all the previous years). * Fixed the bug in util/solve_plate_with_UCAC4 that resulted in sub-optimal choice of stars for computing local astrometric correction. * It is now possible to plot a lightcurve of an object, selecting it on the reference image. After processing a set of images in the usual way (like './vast ../dir_with_images/*') run the new script ./select_star_on_reference_image It will display the reference image with all the detected source marked. You may click on a source to see its lightcurve. Note that some sources detected on the reference image that did not pass selection criteria on the number of detections and extraction flags may not have a lightcurve file written for them (and hence it will not be possible to display their lightcurves). * Updated the script util/search_databases_with_vizquery.sh to make it more suitable for stand-alone use. Suppose you are interested which star is located at the position 18:38:06.48 +39:40:06.0 J2000 and what is its spectral type (guessed from 2MASS colors), just run: util/search_databases_with_vizquery.sh 18:38:06.48 +39:40:05.9 * The lightcurve formatting tool util/cute_lc now sorts the output lightcurve in JD. * Updated the internal copy of SExtractor to v2.19.5. Note that the internal copy of SExtractor is configured with '--disable-model-fitting' option in order to avoid dependency on the ATLAS library. Therefore, it cannot perform PSF-fitting photometry. If you need PSF-fitting photometry - install SExtractor system-wide. * The internal copy of SExtractor is now compiled with the optimization flag '-march=native' if appropriate. * Updated the internal copy of CFITSIO library to v3.390. * Added a copy of FITSVERIFY https://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/ to test that input images are actually valid FITS files. * The CFITSIO example program that prints a FITS header is now available as util/listhead * The CFITSIO example program for editing keys in a FITS header is now available as util/modhead * VaST now creates image*.cat.info files with (more-or-less) detailed information about how each input image was processed. * VaST now creates vast_lightcurve_statistics_format.log file with a plain-text description of the format of vast_lightcurve_statistics.log - the file that contains all the variability indices computed for each star in the field. * Minor bug fixes and improvements. 13 Sep 2016: vast-1.0rc76: * Added the command-line period search tool implementing Lafler & Kinman (1965, ApJS, 11, 216) and Deeming (1975, Ap&SS, 36, 137) methods. The code is shared with the online period search tool accessible at http://scan.sai.msu.ru/lk/ or through the VaST lightcurve inspection tool. The online tool is still recommended for interactive period search. The code distributed with VaST is useful to estimate the upper limit on the significance of a period by performing lightcurve shuffling as described by, for example, Koen & Eyer (2002, MNRAS, 331, 45). Note that the lightcurve shuffling technique gives only an upper limit on the period detection significance as it ignores all systematic instrumental variations that might be present in the lightcurve (the shuffling is based on the assumption that all magnitude measurements are independent). Example usage of the command-line period search tool: lib/deeming_compute_periodogram lib/test/hads_p0.060.dat 1.0 0.05 0.1 this will search the lightcurve lib/test/hads_p0.060.dat for periodicities using the Deeming method in the range of trial periods from 1.0d to 0.05d with the frequency step corresponding to the maximum phase shift of 0.1 between the first and the last points of the lightcurve. The program will print 16.661975 111.832792 DFT where 16.661975 is the frequency (cycles per day) of the highest DFT peak 111.832792 To estimate the significance of this peak one may shuffle the lightcurve 100 times: lib/deeming_compute_periodogram lib/test/hads_p0.060.dat 1.0 0.05 0.1 100 (takes long time to compute as this is a short period searched in a long lightcurve): 16.661975 111.832792 0.000000 +/- 0.000000 DFT here 0.000000 +/- 0.000000 is the reported p-value of the period and it's error (lower p-value corresponds to a more-significant period detection). In this example none of the 100 shuffled lightcurves gives a peak as significant as the one found in the real lightcurve, so the reported p-value = 0.0. Typically, 1000 to 10000 shuffling iterations are needed for a reliable estimation of p-value. The same period search and lightcurve shuffling, but using the Lafler & Kinman method: lib/lk_compute_periodogram lib/test/hads_p0.060.dat 1.0 0.05 0.1 100 will print 16.661975 1.631221 0.000000 +/- 0.000000 LK * The interactive online period search script ( ./pokaz_laflerkinman.sh that can be called from the lightcurve inspection window by pressing 'L') is now not tied to a single period-search server. The script randomly chooses one of a few servers after checking their availability. * Fixed the GCVS search with util/search_databases_with_curl.sh (the output format of the search form has changed which broke down the search script). * The Makefile is renamed to GNUmakefile and the new makefile is intended only to start gmake. This, together with some minor changes in the compilation scripts, should save a user from the necessity to think if 'make' or 'gmake' command should be used to compile VaST on a particular operating system: now it's always 'make' and the scripts will figure out on their own if and how to run GNU Make. However, GNU Make needs to be installed in the system (and accessible as either 'make' or 'gmake') as the current VaST makefiles are incompatible with other make flavors. * Updated tai-utc.dat file includes the leap second to be inserted on 2016 December 31st at 23:59:60 UTC. (This is needed for accurate UTC->TT conversion after that date.) 21 Feb 2016: version vast-1.0rc75: * Minor modifications to speed-up the star matching algorithm. * When manually opening a lightcurve file with './lc' it is now possible to refer to the star only by its number without specifying the full filename, i.e. './lc 12184' instead of './lc out12184.dat'. The old way with the full file name still works, of course. * VaST can now properly handle the situation when DATE-OBS keyword is present in the FITS image header, but does not specify the observing time and a separate TIME-OBS keyword is not present. In this case, VaST will try to get the observing date from JD keyword. * Rewritten the SysRem algorithm implementation (Tamuz, Mazeh, Zucker, 2005, MNRAS, 356, 1466). The new version should work faster and be less aggressive in rejecting uncorrectable stars. The new implementation may be run as util/sysrem2 I'm keeping the old util/sysrem too for now. * Changed the way maximum trial period is set for the period search in pokaz_laflerkinman.sh Now pmax=int(jd_range/3.0-0.5) * Fixed the experimental script that retrieves Catalina sky survey data for a specified (small) sky area: util/catalina_search_area_TEST.sh * Added the interquartile range (IQR) as another measure of lightcurve scatter. In practice it turns out to be very similar to the median absolute deviation (MAD) that was added earlier. * Added a small tool that splits multi-extension FITS files into single-extension FITS images that can be processed with VaST: util/split_multiextension_fits /path/to/my_multiextension.fits * Changed PSF extraction settings in default.psfex The new settings are more appropriate for a wide-field instrument. The old settings may be found in default.psfex.small_FoV It may be a good idea to apply a few iterations of SysRem ( util/sysrem2 ) to remove residual systematic effects arising from imperfect PSF reconstruction that may degrade photometric accuracy for bright stars. * Changes in util/solve_plate_with_UCAC4 to make it work faster. The results of its work are not exactly the same as in the previous version, but should be still acceptable. * Circumvent the mysterious gcc bug that refused to compile VaST with the parameter "-march=native" set on some combination of gcc version and CPU model. The new test is trying to compile the problematic C-source file and does not set "-march=native" if this test compilation fails. * Minor improvements in the Makefile to speed-up compilation. The changes also allowed to fix problem with compilation on some Ubuntu systems. Unfortunately, the changes rely on GNU make syntax, so if you are on non-Linux system, it is now necessary to compile VaST with 'gmake' instead of just 'make'. * Fixed the VaST testing script ( util/examples/test_vast.sh ) that reported a failed test after the variable "NSV 11188" got its GCVS name "V1451 Her" that was not previously recognized by the script. * Increased timeout for plate-solving (both remote and using a local copy of Astrometry.net software). 24 Oct 2015: version vast-1.0rc74 * VaST is now not trying to create a flag image in cases where there are many zero-value pixels scattered across the image rather than being grouped in specific areas (like along a frame edge). This makes it possible again to process background-subtracted images with VaST. * Fixed the bug with incorrect observing time being set from "EXPSTART" keyword. No problem appeared if other keywords ("DATE-OBS", "JD") were present in an image header. * By default, VaST now accepts stars having SExtractor flag set to 1: "The object has neighbors, bright and close enough to significantly bias the photometry, or bad pixels (more than 10% of the integrated area affected)." It appears that in practice many stars that can be measured well get this flag. As always, you may manually specify the maximum acceptable SExtractor flag with '-x' option on the command line: # Accept blended stars ./vast -x3 /path/to/my/images*.fit # Accept everything including saturated stars ./vast -x99 /path/to/my/images*.fit * VaST now compiles with link time optimization (LTO) enabled by default on Linux systems. If this doesn't work for you - remove `lib/set_lto.sh` from OPTFLAGS in Makefile and re--compile with 'make'. * Further improvements to the automatic testing script util/examples/test_vast.sh Beware, the script will download ~2Gb of test data and will need an hour to run! If you have fast internet connection, you are welcome to run this test script. Please write me if the test script reports any problems. * Added warning messages discouraging a user from running VaST on a small number of images (and then be disappointed by the result). Currently, the two 'proper' ways to run VaST are: 1) Run VaST on a long (50-100-1000) series of images and inspect lightcurves that show a large scatter. 2) Run VaST in the 'transient detection' mode on *four* (4) images: two reference and two second-epoch images. Then create an HTML search report by running util/transients/search_for_transients_single_field.sh VaST will run on a different number of images 2, 3, 5 (now printing the warning message), but the results are unpredictable. * Added Median Absolute Deviation (MAD) as one of the displayed variability indexes, see https://en.wikipedia.org/wiki/Median_absolute_deviation * Changed the "Modified sigma" index to the simple "Weighted sigma" * Stetson's J and L variability indexes computed using time-based weighting are now displayed. For a detailed discussion of these indexes see http://adsabs.harvard.edu/abs/1996PASP..108..851S and http://adsabs.harvard.edu/abs/2012AJ....143..140F * S_B variability detection statistic is now displayed, see http://adsabs.harvard.edu/abs/2013A&A...556A..20F for a detailed discussion. * The variability index plots are now displayed in the following order: 1) Standard deviation 2) Weighted standard deviation 3) Median Absolute deviation (MAD) 4) Robust median statistic (RoMS) 5) Stetson's J with time-based weighting of pairs 6) Stetson's L with time-based weighting of pairs 7) S_B variability detection statistic The first four indexes characterize the lightcurve scatter, Stetson's indexes (5 and 6) characterize the degree of correlation between consecutive measurements, S_B statistic (7) combines information about the correlation and the overall scatter. As before, you may switch between these plots by pressing 'M' or 'N' on the keyboard while in the variability search window (./find_candidates). * util/load.sh the script that loads work previously saved with util/save.sh now checks if SExtractor configuration files that are to be loaded are the same as the ones currently present in VaST directory. If the files are identical, the script doesn't bother user with the silly question "Do you want to rewrite a configuration file?". The program md5sum should be installed in your system for this functionality to work. * The VaST settings header file src/limits.h is renamed to src/vast_limits.h to avoid confusion with the standard C library header file also called limits.h * Edited src/gsl/Makefile.in so GSL can compile without makeinfo (texinfo) installed in the system. * Added timeout of 300 seconds to all network communications so the program doesn't hang indefinitely if the connection fails at a critical moment. * Changed the way all VaST subroutines read lightcurve files - now they use a dedicated function rather than simple 'scanf()'. This will eventually allow VaST to natively handle lightcurve files in formats other than the strict VaST format of "JD mag err X Y ap imagefilename". The downside of this change is that lightcurve reading is noticeably slower than it used to be. The support of multiple lightcurve formats is still in the experimental stage though. For now, in order to import ASCII lightcurve files in the format "JD mag err" into VaST use the script util/convert/three_column_ascii2vast.sh which will convert them into the strict VaST format. 23 Jun 2015: vast-1.0rc73 * Fixed the bug that caused VaST to fail if it tries to guess a saturation limit of an image that does not contain BUNIT key in its header (sadly, the majority of ground-based images do not have this keyword). * Saturated pixels with values = 65535 are not flagged-out in the flag image in order not to remove the saturated objects from the list of detections. The brightest objects, even saturated ones, are useful for image matching and performing a blind plate solution with the Astrometry.net code. * An additional aperture photometry run is performed for each image processed in the PSF-fitting mode. The goal is, again, to have the list of detections that includes the brightest stars. Under the normal circumstances, saturated stars are omitted by VaST from the list of detections when working in the PSF-photometry mode. The brightest stars are needed to perform blind plate solution with the Astrometry.net code. * VaST is not attempting to create a flag image for input images that contain many negative values as, in this case, zero-value pixels are likely not the bad ones and should not be flagged. * Restored the partly-broken transient detection mode. See the instructions at http://scan.sai.msu.ru/vast/#transientsearch if you want to try it. The links to external resources in the HTML transient search report located in transient_report/index.html (including MPChecker as well as CSS and NMW archives) are working again. * It is now possible to exit the catalog magnitude calibration plot by pressing 'x' on a keyboard. Before the program was expecting only the capital 'X' or right mouse click to exit. * Added timeout and retry attempts to util/solve_plate_with_UCAC4 subroutine in order to be able to recover from network connection failures during communication with VizieR. * Further improvements to the automatic testing script util/examples/test_vast.sh Beware, the script will download ~1.5Gb of test data! If you have fast internet connection, you are welcome to run this test script. Please write me if the test script reports any problems. * Updates and broken-link fixes on the VaST web page: http://scan.sai.msu.ru/vast/ * The outdated README file is replaces by an HTML page in doc/ directory. 10 Jun 2015: vast-1.0rc72 * Fixed the PSF photometry mode that was not working for images that do not require creation of an external flag image. * VaST is now trying to explicitly set GAIN value if it is found in the FITS header ("GAIN" keyword). You may also set the GAIN value manually in default.sex file - VaST will not try to override it if if it finds that the default gain value in default.sex is changed. VaST will also try to recognize "count" and "countrate" images where the image units are "electrons" and "electrons/s" instead of ADUs (electrons/gain) and set an appropriate gain value ("1" or equal to exposure time in seconds, respectively). The correct gain value is needed to estimate Poisson noise contribution to photometric measurement errors. See equation (61) in Section 10.4 of SExtractor documentation. Note that SExtractor itself will try to override both GAIN and SATUR_LEVEL values set in default.sex using FITS header keys specified in default.sex as GAIN_KEY and SATUR_KEY. Please set an appropriate gain value in default.sex if this information is not available from FITS image header. * The plotting programs ./find_candidates, ./lc, ./pgfv now ignore clicks outside the plotting area, so now one can easily switch between multiple windows by left-clicking green area outside the plot in each window. * The lightcurve viewer ./lc now marks the latest lightcurve point that was inspected by user. * The internal copy of CFITSIO library is updated to v3.370. * Introduced VaST testing script util/examples/test_vast.sh Beware, the script will download ~1.5Gb of test data! If you have fast internet connection, you are welcome to run this test script. Please write me if the test script reports any problems. 27 May 2015: vast-1.0rc71 * Restored the PSF photometry mode. It requires PSFEx and a recent SExtractor version 2.8 or later to work. Unfortunately, PSF fitting cannot be performed with the old Sextractor-2.5 supplied with VaST, so you'll need to manually install a newer SExtractor and PSFEx in your system. Please consult PSFEx manual to learn about PSF extraction parameters set in default.psfex These parameters will most likely require fine tuning for your images. To start VaST in the PSF photometry mode: ./vast -P ../sample_data/f_72-0* * Instead of just displaying the familiar mag-sigma plot, ./find_candidates can now also plot the following variability indexes as a function of magnitude: + Weighted sigma divided by the number of points in the lightcurve. The idea is to artificially hide the lightcurves having large sigma but only a small number of measurements. The weighting scheme follows Stetson (1996) attempting to downweight outlier points in a lightcurve. + Stetson's L variability index as defined in the 1996 paper + Robust Median Statistic (RoMS) as defined by Enoch, Brown & Burgasser (2003) + The simple "peak-to-peak" variability index suggested by Aller, Aller & Hughes (1992) The peak-to-peak index is defined as v = ( ( M_max - error(M_max) ) - ( M_min + error(M_min) ) ) / ( ( M_max - error(M_max) ) + ( M_min + error(M_min) ) ) so it can be negative if the observed lightcurve scatter is lower than estimated errors. Use 'M' and 'N' keys to switch between the plots. For all the plots, a higher value generally indicates that an object is more likely to be variable. * The default SExtractor parameter files are changed to use "XWIN_IMAGE, YWIN_IMAGE" star positions instead of "X_IMAGE, Y_IMAGE". It should result in an improved astrometric accuracy at the cost of slightly longer computing time. See Section 10.2 of the "official" SExtractor documentation for details. * The internal FITS viewer ./pgfv now supports 32-bit images that lack BZERO keyword in the header. * The internal FITS viewer ./pgfv can now handle images with a narrow range of pixel values like from 0.0 to 1.0. * The internal FITS viewer ./pgfv can work with files having a white space in the name. * The internal image viewer ./pgfv, if started from the lightcurve inspection tool ./lc, will automatically zoom-in on the star of interest. You may see the full image by pressing 'D' or 'Z'+'Z' to zoom-out. * Renamed ds9 region file from vast_list_of_all_stars.ds9 to vast_list_of_all_stars.ds9.reg in order to prevent ds9 from trying to open it automatically each time it starts. The proper way to use this file would be ds9 /path/to/reference/image.fits -region vast_list_of_all_stars.ds9.reg * VaST now supports FreeBSD operating system. Tested on FreeBSD 10.1 with gcc. Changes are introduced in an attempt to allow VaST to handle HST Level 1 (exposure) images from the Hubble Legacy Archive: * Image date can now be set from EXPSTART key containing MJD(UTC) of the exposure start. * VaST is trying to guess the saturation limit for each image as guessed_saturation_limit=maxval-SATURATION_LIMIT_INDENT*maxval where maxval is the brightest image pixel and SATURATION_LIMIT_INDENT is set in src/limits.h. By default SATURATION_LIMIT_INDENT=0.1. This behaviour may be turned on by running ./vast with -g option or off by using -G option. If none of the two options is specified explicitly, VaST will make a wild guess if it should use SATUR_LEVEL specified in default.sex or if it should try to guess it for each image individually. * VaST will check how many pixels having the value of exactly zero are in the input image and if there are more than FRACTION_OF_ZERO_PIXEL_TO_USE_FLAG_IMG*total_number_of_pixels the program will automatically produce a flag image for SExtractor to filter-out stars overlapping with these bad pixels. The value of FRACTION_OF_ZERO_PIXEL_TO_USE_FLAG_IMG can be changed in src/limits.h. Make sure to run 'make' after updating src/limits.h for the changes to take effect. 11 Mar 2015: vast-1.0rc70 * Automatic choice of a VizieR mirror. * VaST can now handle filenames with white spaces if they are specified in the command line (but not in the vast_list_of_input_images_with_time_corrections.txt file, sorry). * VaST is trying to guess the field of view of an image (for plate solving) if it was not specified explicitly. The guess is based on a previously-solved image of this same field or on the FOCALLEN key in the FITS header (assuming it contains focal length of the telescope expressed in mm). * If a blind plate solution failed with a specified (or guessed) field of view, the program retries to solve the image assuming a larger or smaller field of view. 7 Mar 2015: vast-1.0rc69 * The program can now again work with VizieR. * Astrometric calibration of images can now be performed using one of a few plate-solving servers (the best one is be chosen automatically) or using a local copy of Astrometry.net software (if it is installed in the system). * Improvements in the plate solving algorithm. * VaST FITS image viewer (./pgfv) now remembers image orientation (set by pressing 'V' and 'B' on a keyboard) after changing zoom (keyboard 'Z'). * The magnitude-scatter plotter if started in the non-recompute mode (./find_candidates aa) will now remember stars viewed by user earlier (indicated as green points on the plot). * While performing a manual magnitude scale calibration with util/magnitude_calibration.sh it is now possible to cancel the input by typing 99 instead of an actual star magnitude. This is useful in case one selected a wrong comparison star. * The magnitude calibration routine is now trying to guess if parabola or the "photocurve" provide a better approximation of the "measured mag."->alog mag." plot.elation. One can always override the automatic choice by pressing 'P' while looking at the "measured mag."->"catalog mag." plot. * Fixed the bug that prevented lib/vizquery from working properly if cdsclient is not installed system-wide (wwwget was missing from $PATH). Now VaST includes a fully self-contained copy of cdsclient. * VaST now supports images obtained with the SBG camera of Kourovka Astronomical observatory (Ural Federal University) that have a non-standard FITS-like header. * Updated tai-utc.dat file includes the leap second to be inserted at the end of June 30, 2015 at 23:59:60 UTC. (This is needed for accurate UTC->TT conversion after that date.) 17 Jun 2014: vast-1.0rc68 * The program can now again access VizieR. This feature was broken because of the change in order of output columns printed by vizquery script. Star identification scripts now work again. :) * The Kwee - van Woerden (1956BAN....12..327K) minimum finding method built into the VaST lightcurve viewer (./lc) now returns the full JD, not the one relative to the lightcurve start. To activate the Kwee - van Woerden algorithm, press 'K' in the lightcurve inspection window. * The input images may now be specified not only as command line arguments, but also listed in the file vast_list_of_input_images_with_time_corrections.txt located in the VaST directory. The file should have two columns: absolute or relative path to the image file and time correction in seconds that should be applied to the file (in most cases should be 0.0). Full JD of the image may also be specified instead of the time correction. That will override JD determined from the image FITS header. VaST will guess what it is based on the specified value. See examples in vast_list_of_input_images_with_time_corrections.txt_example * Instead of specifying each image on the command line or in the vast_list_of_input_images_with_time_corrections.txt file, it is now possible to give a directory name as a command line argument. VaST will process all FITS images found in the specified directory, but it will not look recursively into sub-directories. Note that if only the directory name is specified, user has no control over which image in this directory is processed first and, therefore, used as the reference image. It is possible to mix all three input methods (individual images specified in a command line, vast_list_of_input_images_with_time_corrections.txt file and directories specified in a command line) in a single VaST run, so one can specify the first (reference) image explicitly on the command line. * Fixed the bug causing segmentation fault when input images have different size. Note that the input images may have different size in pixels, but still have to have the same scale ("/pix) in order to be processed in a single VaST run. * Minor improvements in the image matching algorithm resulting in a ~10% speed-up on a typical dataset. 13 May 2014: vast-1.0rc67 * Fixed the bug that caused "buffer overflow" crash on long image series (of a few thousand images). * New routine provides a locally-corrected astrometric position for each object on an image. The astrometry is based on UCAC-4. The routine may be called directly as util/solve_plate_with_UCAC4 myfitsimage.fits 40 (here myfitsimage.fits is the FITS image file name, "40" is the approximate field of view in arcminutes) to produce a list of sources on the given image. A better way to benefit from it is by using the automated identification script (util/identify.sh or 'U' key in the lightcurve inspection window) and (hopefully) enjoying the much-improved astrometric accuracy. This new astrometric routine needs Internet connection to work, but the identification script needs it anyhow... * It is now possible to use APASS magnitudes listed in UCAC-4 to calibrate the instrumental magnitude scale. Use the script util/magnitude_calibration.sh R 40 (where "R" is the observing band/filter and "40" is the approximate field of view in arcminutes passed to the plate solution routine). The choice of observing bands is the following: + "C" for "clear" are the unfiltered UCAC-4 magnitudes + "B" are the APASS B magnitudes + "V" are the APASS V magnitudes + "r" are the APASS r magnitudes + "i" are the APASS i magnitudes + "R" are Rc magnitudes computed from APASS V, r and i following Jester et al. (2005), see http://www.sdss.org/dr4/algorithms/sdssUBVRITransform.html Choose the scale that best matches your observing band. "R" is the usual choice for unfiltered CCD chips, "V" may be appropriate for unfiltered blue-sensitive CCD chips and green-channel of DSLRs, "B" may be good for blue-sensitive photographic plates. The old version of the magnitude calibration script is renamed to util/magnitude_calibration_USNO-B1.0.sh * Internal copies of libraries are updated to CFITSIO v3.360 and gsl-1.16. * Plate solution details (such as field of view size and its center) are now printed out in a terminal. * Improved Aladin script that generated and launched if you have Aladin installed locally. The script now displays both DSS and the original input images. This may be useful to visually confirm identification of an interesting object. The star suggested as a correct identification by the automated procedure is clearly marked in the Aladin plot. * Server-side plate solution software is upgraded to astrometry.net-0.46 which (in a combination with new index files produced from Tycho-2 and 2MASS catalogs) provides considerably better solutions of some badly distorted images (think telephoto lenses and wide-field photographic plates). Silly enough, it turned out that previous versions of the VaST plate solution routine cannot handle the long FITS header produced by astrometry.net-0.46. Please upgrade to vast-1.0rc67 if you conduct plate solutions from within VaST. 27 Mar 2014: vast-1.0rc66 * Fixed the script performing absolute magnitude calibration (util/magnitude_calibration.sh). It was broken by a change in the output format of vizquery. To calibrate the magnitude scale using USNO-B1.0 B magnitudes as a reference you may execute: util/magnitude_calibration.sh B to do the same using R magnitudes from USNO-B1.0: util/magnitude_calibration.sh R * Fixed a problem preventing VaST from compiling on some virtual machines when "gcc -march=native" cannot correctly recognize the processor architecture. * Fixed a problem preventing VaST from starting on a system that is low on RAM. Now, if there is enough memory to process the data - VaST should be able to use it efficiently. * New log files vast_list_of_all_stars.log and vast_list_of_all_stars.ds9 contain pixel coordinates of all detected stars in thee list formated as a region file that can be loaded into the ds9 FITS image view list wile vast_list_of_all_stars.ds9 is the same list formated as a region file that can be loaded into the ds9 FITS image view list wile vast_list_of_all_stars.ds9 is the same list formated as a region file that can be loaded into the ds9 FITS image viewer. 29 May 2013: vast-1.0rc65 * Added linear magnitude correction as a function of star position on a frame. This is useful to correct extinction on very wide field (a few degrees) images such as those obtained in the NMW survey ( http://scan.sai.msu.ru/nmw/ ). By default, the program will turn on this correction if the reference image has more than 10000 stars on it. You may manually turn this correction on (off) by starting VaST with "-j" ("-J") flag. If you have wide field images, try to experiment with this option and see what works best for your data. * The star matching algorithm is now allowed to use saturated and blended stars for image identification. This slightly degrades the quality of the initial plate solution, but that is corrected for at a later stage. The good thing is that this change makes the algorithm much more stable and it typically requires less reference stars to match images. The image matching process becomes faster. * Fixed broken links to WISE-Atlas in the transient detection mode. * Fixed the lightcurve viewer (./lc) crash when one tries to delete the first point from a non-VaST-formatted lightcurve. 13 Apr 2013: vast-1.0rc64 * "Official" support of the transient detection mode. The code was there for quite some time, but was lacking documentation and had to be tested extensively. The code is still FAR from perfect, but I believe it may already be useful. See the VaST home page for details. * A major bug that prevented stars with very large sigma from appearing on the sigma-magnitude plot is now fixed. Thanks to Fidrich Robert for discovering it! * "JD" keyword is read form a FITS header only if "DATE-OBS" keyword was not found. Useful for FITS files that have both keywords in their headers. * Fixed an issue that prevented VaST to compile on macOS after the latest development environment update. Thanks to Alexei Alakoz for his valuable help in testing! * A new script util/catalina_search_area_TEST.sh is introduced that allows to search Catalina Sky Survey data in a small area around a particular sky position. The data may be processed using standard VaST tools. The current implementation of the script is very slow and is meant only for testing and feasibility studies, it is not ready for mass-processing of CSS data. * An automatic-testing script lib/test_vast_release.sh is introduced. It currently works only with a system-wide installation of SExtractor 2.8.6 and consumes tens of megabytes of Internet traffic. But, if this is not a problem - you are welcome to try it! * Minor bug fixes. 16 Jan 2013: vast-1.0rc63 * Fixed the major bug that caused the new star matching algorithm (introduced in vast-1.0rc62) to fail to match some stars in close pairs. The problem was evident on large datasets when the program got swamped by unidentified stars, while the older versions of VaST could handle these datasets. I am deeply grateful to E. G. Lapukhin for spotting the problem. Thank you! :) * The program now uses internal versions of PGPLOT and GSL libraries, so users do not have to worry about installing these libraries system-wide. * VaST now comes with a built-in SExtractor v2.5. It is still recommended to use a system-wide installation of a newer SExtractor version, but now, if VaST fails to find "sex" executable in the $PATH, it will use the internal one. This might be helpful for those who have problems compiling new SExtractor or, more likely, ATLAS libraries on which the new SExtractor depends. * Individually processed images may now be viewed with DS9 instead of the internal VaST image viewer: ./sextract_single_image -9 image.fits * Fixed the bug that caused VaST not to create vast_stars_with_large_sigma.log file when used with gsl-1.15. * VaST was successfully tested on MacOS X 10.8.2 'Mountain Lion' + XQuartz, Xcode, MacPorts -> gcc (including gfortran and g++), wget (resolution is fine with a retina display). * Internal CFITSIO library upgraded to Version 3.31. Overall, if one has a Linux or Mac system with a working development environment based on gcc, the new version of VaST is supposed to just compile and work, with no need to manually install additional libraries and programs. How well does it work in real life? Well... Bug reports and success stories are welcome, as always! ;) 28 Oct 2012: vast-1.0rc62 * Improved star matching algorithm resulting in ~2x overall performance increase on large datasets. * The online period search interface ("L" in the lightcurve inspection window) tries to set reasonable maximum and minimum trial periods automatically. * All lightcurves inspected with the online period search service are saved locally in the "saved_period_search_lightcurves/" directory, so it's easier to find them later. 28 May 2012: vast-1.0rc61 * The new experimental feature in vast-1.0rc61 is its integration with the online period search service. You may try it by pressing 'L' key at the lightcurve inspection window (./lc). The lightcurve will be sent to a remote server for analysis and a browser window will open displaying the results. Naturally, this feature requires an active Internet connection to work. You are also welcome to try the period search service as a stand-alone web-application by pointing your browser to http://scan.sai.msu.ru/lk/ * The bug is fixed which caused VaST to crash on images that have no comment field for the DATE-OBS key in the FITS header. * Improved stability of the magnitude calibration solution in "difficult cases". * The new version contains a local copy of the vizquery script from cdsclient-3.7 package. No need to install cdsclient, for the automatic star identification (e.g., "util/identify.sh out01234.dat") and magnitude calibration (e.g., "util/magnitude_calibration.sh R") scripts to work. * VaST correctly handles the situation when it cannot create child processes (for example, because the system runs out of memory) during the execution of the main program. * vast-1.0rc61 compiles on the latest version of Ubuntu. Please check also the updated instructions for Ubuntu users at the VaST homepage: http://scan.sai.msu.ru/vast/#down Frequently Asked QuestionsHow to perform differential photometry with VaST? What are the photometric errors reported by VaST? How to obtain a lightcurve of an individual object with VaST? VaST doesn't detect my target source. What's wrong? VaST doesn't run complaining that no images are found. Which Linux distribution is suitable for VaST? Is there a Windows version of VaST? VaST cannot match images in the input series. How to minimize the detection of stars near the edge of the field? What exact metadata from the FITS image VaST needs, other than the image itself? Exposure time? VaST calibrates the magnitude scale as a part of data processing... How exactly is this done? Is aperture photometry that VaST does enough in most cases, or should I do PSF-fitting photometry? Compilation on macOS fails with "Undefined symbols for architecture x86_64:" How to specify a reference image? How to resize the PGPLOT windows? How to use VaST from behind a proxy? Why do many columns in vast_lightcurve_statistics.log are filled with 0? VaST automatically analyzes only those stars which have at least a minimum of N_max/2 measurements? How to detrend a lightcurve with VaST? How to acknowledge VaST in a publication? List of publications which make use of VaSTThe list includes both refereed and non-refereed publications (conference proceedings, electronic telegrams). If you are aware of a VaST-related publication that should be listed here, please drop me a line at kirx[at]scan.sai.msu.ru
Mike Rosseel's vast-automationThe vast-automation code developed by Mike Rosseel provides an alternative way to identify variable stars detected with VaST and prepare the information for submission to the VSX and AAVSO databases. The code is in python and uses the official Astrometry.net servers for plate solution. Requires a local copy of UCAC4.Variable star search/photometry software not related to VaSTIf VaST doesn't suit your needs, you may be interested in trying other software (listed roughly in the order of decreasing similarity to VaST in design goals and implementation):
Other tools that might be useful
If you have any questions fell free to contact VaST authors via e-mail kirx[at]scan.sai.msu.ru
Good luck in using VaST! VaST developers team. | ||||||||||
VaST developers team, 2005-2024
|