p3d README Version information: This is the release version 1.1 of p3d. All parts of the tool work, although some parts require precise operation in order to work as intended. $Id: README 159 2010-04-16 14:26:46Z christersandin $ Table of Contents: ------------------ 1.0) Introduction * 1.1 About p3d * 1.2 Installing p3d 2.0) Contact * 2.1 Funding acknowledgments * 2.2 Collaborators * 2.3 Reporting Bugs * 2.4 Feature Requests 3.0) Software Platform * 3.1 Coding Philosophy 4.0) Supported Instruments 5.0) Features 6.0) Recompiling the binary metafile for the IDL Virtual Machine 1.0) Introduction: ------------------ 1.1) About p3d: --------------- 1.2) Installing p3d: -------------------- Installing p3d requires a few steps depending on if you only intend to use it with the IDL Virtual Machine (see 3.0) or if you want to use it and develop it using your own IDL license. In any case you need to have IDL, version>=6.2, installed on your system. IDL can be downloaded from the following web page: http://www.ittvis.com. See the relevant IDL documents on how to install IDL, and on how to acquire and setup an IDL license (if you require one). i) Acquiring the p3d software You can either download one of the prepared releases of p3d from the program files area (https://sourceforge.net/projects/p3d/files/) or you can download the latest bleeding edge version using subversion. The latter approach is preferred if you want new functionality, which has not yet been included in the released versions. Although note that there is no guarantee that the code of subversion will be bug free (it is being corrected as time permits). In order to download p3d from the subversion repository at sourceforge.net the first time you first open a terminal and change directory to where you would like to have the software. Thereafter you type the following svn checkout https://p3d.svn.sourceforge.net/svnroot/p3d/trunk p3d whereby all files are placed in the directory 'p3d/' (you can of course use any name on this directory). In order to update p3d you should at any time change directory to p3d/ (or whatever name you have used) and type 'svn update'. For any additional functionality of subversion please see its web page http://subversion.tigris.org/. ii) Using the IDL Virtual Machine There is a (bash-)script provided to run p3d using the IDL Virtual Machine on all UNIX-type platforms. Although, before you can use the script you have to prepare your system in a few steps. After you have installed IDL you have to setup your environment to tell the system where it can find IDL. The environmental variable IDL_DIR must be set to the name of the directory where your IDL installation resides. Depending on your system shell you can add the following line to your ~/.bashrc (bash shell): export IDL_DIR=/path/to/IDL/ or ~/.cshrc (C shell): setenv IDL_DIR /path/to/IDL/ where the tilde character (~) denotes your home-directory. If you have python installed (with the name 'python') the p3d startup script will check if you have the right version of IDL installed (>=6.2). The environmental variable p3d_path must be set to the name of the directory where you have copied the p3d distribution. Additionally you might want to set the environmental variable p3d_data_path to the name of the (base) directory where you keep the data that you will work with. You may also want to add the vm/ directory to you path variable so that you don't have to enter the full path to where p3d resides every time you want to run it. Depending on your system shell you can add the following lines to a suitable place in your ~/.bashrc file (bash shell; Bourne Shell type environment): export p3d_path=/path/to/p3d/ export p3d_data_path=/path/to/IFU/data/ # Optional export PATH=$PATH:${p3d_path}/vm # Optional or ~/.cshrc (C shell): setenv p3d_path /path/to/p3d/ setenv p3d_data_path /path/to/IFU/data/ # Optional setenv PATH $PATH:${p3d_path}/vm # Optional In order to start p3d you thereafter enter the following in a terminal (if you added ${p3d_dir}/vm to your system path): p3d_vm.sh or, if you didn't add ${p3d_dir}/vm to your path: ${p3d_dir}/vm/p3d_vm.sh Note! The bash-script p3d_vm.sh checks if the required IDL binary meta file p3d.sav is present in the ${p3d_path}/vm/ directory. If it exists IDL is started using that binary file. Note! If you have an IDL license, and want to recompile the binary for use with the IDL Virtual Machine you can use the provided script ${p3d_dir}/vm/p3d_tool_makevm.sh. See below (6.0) for further instructions. iii) Using an IDL license In order to run p3d using a license it is necessary to tell IDL where it can find all required source files. In addition to all files of the p3d distribution the files of the following sources are required: Astro-lib (http://idlastro.gsfc.nasa.gov/homepage.html) All files (actually a subset, but downloading all is easier [for us]). Mpfit (http://purl.com/net/mpfit) mpfit.pro and mpcurvefit.pro These files can be put anywhere, but if you would like to use the provided scripts when generating a binary metafile for use with the IDL Virtual Machine (if you made any changes to the code) it is recommended to download these files to ${p3d_path}/contrib/astro-lib (for the files of Astro-lib) and ${p3d_path}/contrib/ (all other contributing files). In order to download the files to these directories automatically use the provided script: ${p3d_path}/vm/p3d_acquire_xtools.sh When all files are downloaded, or if they are already present on the system, it is necessary to setup IDL to find these files. The most straightforward way is to set the environmental variable IDL_STARTUP to the name of an IDL program where the necessary information is defined. The following lines can be used to do this (the filename IDL_STARTUP.PRO can of course be chosen freely): For a Bourne Shell type environment, add these lines to your ~/.bashrc export IDL_STARTUP=/path/to/IDL_STARTUP.PRO # Modify to fit your system export p3d_data_path=/path/to/IFU/data/ # Optional For a C Shell type environment, add these lines to your ~/.cshrc setenv IDL_STARTUP /path/to/IDL_STARTUP.PRO # Modify to fit your system setenv p3d_data_path /path/to/IFU/data/ # Optional Thereafter add the following lines to IDL_STARTUP.PRO: astrolib_path='/path/to/astrolib/root/directory' # Modify to fit your sys. markwardt_path='/path/to/markwardt/files/directory' # Modify... p3d_path='/path/to/p3d' # Modify to fit your sys. !path=markwardt_path+':'+astrolib_path+'/pro:'+!path !path=p3d_path+':'+p3d_path+'/routines:'+!path If you chose to put the additionally required files under the root directory of p3d, as we recommended above, you could add the following lines to IDLSTARTUP.PRO instead: p3d_path='/path/to/p3d' # Modify to fit your sys. !path=p3d_path+'/contrib:'+p3d_path+'/contrib/astro-lib/pro:'+!path !path=p3d_path+':'+p3d_path+'/routines:'+!path Note! The exact order that paths are specified in is important. Also, observe that no extra directory separator ('/') should be used in the final position of the specification of the paths (i.e. '/path/to/p3d' not '/path/to/p3d/'). p3d can now be started from the idl prompt using the command IDL> p3d If you'd like to know the available options you could just have a look in the program file (using your favorite editor, or e.g. more or less), or type: IDL> p3d,/help 2.0) Contact: ------------- All collaborators, unless noted otherwise, have the following affiliation: Astrophysikalisches Institut Potsdam (AIP) Instrumentation An der Sternwarte 16 DE-14482 Potsdam GERMANY Tel: +49-(0)331-74990 http://www.aip.de p3d project web site: http://p3d.sourceforge.net 2.1) Funding acknowledgments: ----------------------------- This project has been made possible through funds from the following sources: BMBF: 03Z2A51, 05A08BA1 2.2) Collaborators: ------------------- Lead Programmer: Christer Sandin o If you have comments of bugs or feature requests, which are directly related to p3d, and that you cannot add to the tracker at the project web site (see below for instructions) then please send them to: Christer Sandin Additional Programming: Joris Gerssen o Improved the treatment of the wavelength calibration in order to make it more intuitive than it used to be. o Included support for reading and writing output in the e3d-format. o Started the work to write code to fit profiles for the optimal extraction. Peter Weilbacher o Included an option to calibrate the dispersion mask against sky emission lines. Testing of Software: Ana Monreal-Ibero (ESO) PMAS/LARR Original Programmers/Authors: Thomas Becker o Thomas wrote the first version of many parts of p3d for a large number of instruments. This public release version is based upon the earlier variant of p3d that was called p3d_online. Petra Böhm o Maintained all previous versions of p3d (including p3d_online, ppak_online, virus_online, p3d, etc.), she also fixed bugs and wrote various supporting documents. Martin Matthias Roth o Martin worked with software design of early versions of p3d. Additional acknowledgments: Luzma Cairós o Tested the dispersion mask creation tool thoroughly and provided many tips on its improvement. Ole Streicher o Has provided p3d with a first version of the tutorials at the project web site. Stefano Ciroi (Padova) o Figured out how to setup a basic configuration of p3d to be used with the BTA/MPFS (3kx2k CCD) IFU. 2.3) Reporting Bugs: -------------------- In order to report any kind of bug we urge you to use the bug tracker that is available at the p3d project web site (Develop/Tracker/Bugs). This way your bug will be seen by all users and programmers - and it won't be forgotten. At the bug tracker page you can search previously reported bugs, to see if your bug has already been reported. Or you can open your own bug description (by clicking on "Add new"). Doing this, please make sure you are using the latest version on the subversion trunk of p3d (or specify the version of the release you are using). Thereafter specify a category for the bug, and provide a short summary of the problem, and most importantly, describe the problem with enough detail that somebody who does not use your setup and data files is able to follow your arguments. Bug reports without a useful description will be assigned a low priority, or even be deleted. If you think it would help us, attach all relevant data files to your report (please compress fits-files using tar/gzip or zip to save space). Additionally, please specify a reasonable priority to your bug. While our intention is to correct serious bugs, which prevent p3d from producing spectra, as soon as possible, we will assign GUI- related issues a lower priority. Important note. When reporting a bug, turn on the debugging mode (/DEBUG) of p3d and paste the resulting output from the IDL terminal/window (once the bug has been reached) into your bug description. The reason we ask you to do this is that the information that is returned by the error handler when the debug keyword is unset only provides the name of the file where the problem occurred, and an IDL-specific number. Such sparse information is much more difficult to interpret than a full output. 2.4) Feature requests: ---------------------- If p3d is missing some functionality that you would very much like to have included then you can make a feature request using the tracker on the p3d project web site (Develop/Tracker/Feature request). Following the description on how to add a new bug description we ask you, also in this case, to be as detailed as possible. Please understand that we are a team with limited resources. If your request concerns a huge modification, which requires many days, or even months, to code it might not happen at all. On the other hand, we are highly interested in taking on project members who are interested in contributing to p3d with their own time and coding. Please drop us a note if you think this could apply to you. 3.0) Software Platform: ----------------------- p3d is fully based on the language of the proprietary tool Interactive Data Language (IDL; cf. http://www.ittvis.com) and cannot be used if IDL is not installed on the system. Although it is necessary to have an IDL license in order to modify code, p3d can be run without an IDL-license using the supplied binary metafile ('vm/p3d.sav'). In this case the so-called IDL Virtual Machine is used to run p3d. p3d has been implemented in such a way that it can be used with full functionality with the Virtual Machine. Since p3d is based fully on IDL it can be used on any platform that is supported by IDL. Note, however, that the installation procedure differs for an installation on a Windows machine compared to that on the other platforms supported (the windows version of the installation instructions is not yet implemented). 3.1) Coding Philosophy: ----------------------- p3d is a large, and growing, code. In order to keep it manageable we have applied rules of coding practice; use of this practice has so far made debugging easy as bugs are found quickly. These are the rules, which we would ask any project programming member to follow: o All files must have filenames which conform to: p3d_task_name.pro o Code layout + All files must have a fully explained header. + All routines use the options: compile_opt hidden,IDL2 + The maximum number of character columns per line is 79. + All blocks are indented by 2 characters. + Comment lines begin with ';;' + Integers are always specified as long with L, e.g. '0L'. Exceptions: VERBOSE and ERROR variables use integers. + All commands and statements are written with small characters. o Common blocks are only allowed for variables which obviously will never ever change, i.e. pretty much never. o An error handler is setup for every routine, unless the user keyword DEBUG is set. o Logging comment statements must always use 'p3d_misc_logger'. o Input parameters should be checked for consistency in terms of type, size, value and range, as far as possible. o All command keywords are for clarity specified in full, i.e. no abbreviations (for example, /XSTYLE not /XST). o Text widgets must include error checking to ensure that the entered values are of the right type and range. o All widgets in GUI tools with a status line must include the UNAME field that specifies what the widget in questions does. Issues related to the subversion repository of p3d: o No file on the trunk should be committed before the modifications are confirmed to work as intended. o All commits to the repository should include a describing message: Example: svn ci --message "Corrected/modified this and that." o Larger modifications should use a branch, and not the trunk. o The idea is to copy the trunk to branch/ in preparations for a release. Then we only make bug fixes on that branch before we copy the final release version to tags/. 4.0) Supported Instruments: --------------------------- We have developed p3d to make it a general tool. General in the sense that it can be used with any Integral Field Unit (IFU). However, before p3d can be used with any instrument it must be properly setup. p3d has been setup, tested, and found to work well with the following instruments: ---The following needs editing!--- PMAS - is mounted on the Calar Alto 3.5m telescope PMAS :: http://www.aip.de/groups/opti/pmas/OptI_pmas.html Calar Alto :: http://www.caha.es LARR (Lens Array) :: 16x16 square-shaped elements that is used with either the 0.5"x0.5" or the 1.0"x1.0" magnification settings. PPAK () :: 331 circular elements, each of 2.8" diameter. Since October 2009 PMAS has been equipped with a new 4kx4k CCD, replacing the previous 2kx4k chip. p3d supports data of both CCDs. In order to use either setup you can select them in the menu upon startup of p3d, or use the input parameters: /larr2k, /larr4k, /ppak2k, or /ppak4k. VIRUS-P - is mounted on the Harlan J. Smith 2.7m telescope McDonald Observatory :: http://www.as.utexas.edu/mcdonald/facilities/2.7m/2.7.html VIRUS-P is a prototype for the VIRUS instrument that is made for the Hobby Eberly Telescope. VIRUS-P (bundle 2) :: has 246 circular spatial elements VIRUS-P (bundle 1) :: has 247 circular spatial elements SPIRAL VIMOS FLAMES/ARGUS IMACS GMOS ... 5.0) Features ------------- 6.0) Recompiling the binary metafile for the IDL Virtual Machine: ----------------------------------------------------------------- The simplest way to (re-)compile the binary metafile that is used with the IDL Virtual Machine is to use the provided scripts in ${p3d_path}/vm/. Note that in order to recompile p3d you must have an IDL license. You could use the following steps from a terminal: i) Set the p3d root directory and IDL_DIR environmental variables. In a Bourne Shell type environment (e.g. bash) type: export p3d_path=/path/to/p3d/ export IDL_DIR=/path/to/IDL/ # IDL version >=6.2 is required! In a C Shell type environment (e.g. csh) type: setenv p3d_path /path/to/p3d/ setenv IDL_DIR /path/to/IDL/ # IDL version >=6.2 is required! ii) Download the required auxiliary files. You could, most conveniently, use the provided script: ${p3d_path}/vm/p3d_acquire_xtools.sh Note! The binary metafile generation script requires the additional files to be present in ${p3d_path}/contrib/astro-lib (with all .pro-files in ${p3d_path}/contrib/astro-lib/pro/) and ${p3d_path}/contrib/. iii) Change directory to ${p3d_path} and generate the binary metafile ${p3d_path}/vm/p3d.sav. You could, most conveniently, use the provided script: pushd ${p3d_path} > /dev/null ${p3d_path}/vm/p3d_tool_makevm.sh popd > /dev/null Note! The binary metafile generation script must be executed from the p3d directory ${p3d_path}! Note! If there is any problem in the binary metafile generation process please check the output of the scripts to find out what the problem is.