Teams / DSLR Documentation and Reduction / Figured out Air Mass Corrections
*Sky & Telescope Illustration by Casey Reed
Privacy Policy | Contact Us | Affiliates | About Us | Participant Sites
International Year of Astronomy | American Association of Variable Star Observers (AAVSO)
EDIT: Upps.. this got posted to the wrong place of the thread...sorry
Okay, I've finished implementing the reduction routine in Excel for the "Intermediate"level reduction. The cool thing is that first-order air mass corrections can be done using a single image so we need not observe stars over a wide range of air masses (although that should be done as a check). I think for the"Advanced" reduction we should simply present the mathematics behind everything and let our participants make their own software/calibration routines. Because of this, I don't think I'll try to add in secondary air-mass corrections to an Excel file (plus they are fairly constant over the course of a year according to Arne's book). I've attached the Excel file that actually performs the reduction using a least-squares fit to the calibration stars. The more cal stars over more air masses, the better. The worksheet is very similar to the first spreadsheet with the exception that air mass columns have now been added and the relavant correction quantities are now computed in a different way. I've left some sample data Tom sent to me last week for a demonstration that it actually works. Please give it a shot and let me know how it goes. I'm in the process of writing up the documentation for the main CS website and something that explains how the computations are done in the Excel spreadsheet. For the website I'm going to recommend that our participants use the AAVSO air mass calculator (or Richard's built-in routines for AIP4WIN once the beta version goes public). Because the air mass can vary by a modest amount between/during exposures, I think we should either (1) instruct our participants to stack/average their data (for low air masses) or (2) reduce each image individually, and averaging the results. What do you all think? 2010-Aug-9 Edit. I have attached a newer version of the spreadsheet to this message including the changes I mentioned on this day. 2010-Aug-9 v2. This version includes semi-automatic air mass calculations. The user only need to supply the coordinates of the objects, the location at which the image was taken, and the local sidereal time.
Hi! I've added some more automation plus a bug fix in the drafts below: 1) airmass for eps aur itself and the comparison star is now computed automatically 2) the sheet now allows to re-use the interpolated coefficients for a series of frames, but will re-calculate airmass for eps Aur and the check star for every frame individually based on the time of the exposure. I think the final version should have all the intermediate results and lookup cells moved to a second page and an instruction page added, as in Tom's draft. All, please let me know what you think about this CS HB
I like it! I'll post this to the CS website during the conference this week, hopefully before the team talk! Thanks for finishing the spreadsheet Heinz!
Hi Brian. Do you had a chance to look at my spreadsheat? I think mine is a bit more flexible in adding and removing compairsion stars. Air mass is also automatically calculated from the stars altitude. The altitude must though be entered but I think this is an info that is easier to find out(from Stellarium or any other planetarium program). I don't use the concept of check star as all compairsion stars act as check stars in the error analyzing part. You can then get an idea of how good the instrumental magnitude was for each of the compairsion stars. I use the Excel bultin function REGR for calculating the coefficients that I think gives a cleaner view of the calculation and is easier to follow.
Okay, I've finished a semi-full implementation of the air mass calculations. The user needs to provide the target coordinates, location at which the image was taken, and the local sidereal time at the moment the image was taken. Best of all I only had to add one intermediate table and one table for the air mass constants from Young's table! I figure several people out there have some planetarium software, therefore they can get the JD and LST from there or one of several online calculators. Please let me know what you think. Brian Edit: I suppose we could ask the person who runs this website: http://www.csgnetwork.com/siderealjuliantimecalc.html if we could use their code as it does both the JD calculation and the LST all at once.
Hi Brian, Maybe for our purposes, LST can be computed from the simple formula for Greenwich Mean Sidereal Time given here: Wiki entry for Sidereal_time#Definition quote:
where D is the interval, in days including any fraction of a day, since 2000 January 1, at 12h UT end quote Then LST = (GMST + Long + 1.00274 * UT) I put that into your latest Excel sheet (I hope Open Office didn't mess it up...but you will get the idea). It yields an LST within a few seconds of your example results. One can always put a better formula in cells K5 .. N5 I guess the layout is not optimal, LST could be moved to the right half of the sheet as it is computed now. What do you all think? CS Heinz
We got the same idea!
Heinz, I stumbled upon this simple conversion from JD to LST the other day, but I was missing the last term to compute LST. I think you've got it right, we should do this instead. I'll include this note in the tutorial and look up the necessary references and disclaimers on it's accuracy (0.1 second / century... probably okay) Thanks! Oh, about using OpenOffice (I'm running Ubuntu all of "my" computers, wife's machine not included). I did the entire spreadsheet in Calc without touching Excel so (at least with a new version of OO) the conversion seems to go well. I'm happy with these revisions, I think we've greatly improved upon the spreadsheet. So, with this addition do you all think we can send it off for general use? Brian
Hi! >So, with this addition do you all think we can send it off for general use? Upps...the automatic airmass calculation for eps Aur itself and the check star (omega Aur) is still missing (last lines on the sheet). That needs to be added (I could do that tommorow), and then I guess we'd merge this sheet with Toms to get a best of both sheets maybe. CS heinz
Actually I don't believe the 0.1 seconds thing because a formula that simple cannot take care of nutation effects, right? But it will be good enough I'm sure. EDIT:I found a formula taking nutation into account here : http://www.mathworks.com/matlabcentral/fileexchange/28232 CS Heinz
Second reply for the day. I've attached an updated spreadsheet using linest (see my original post at the top of this thread). After reading a comment Imade on the "number of cal stars" thread that Heinz started, I decided it would be prudent to have a way to quality check the fit. This is implemented in a table in the spreadsheet. I also included the error in the fit parameters as well as the fractional error (should be < 0.1 for a "good" fit) so our particpants can double-check their measurments. I might also mention that the data listed here is from Charlie, truely excellent photometry! What do you all think?
Greetings Thomas, I have looked at your spreadsheet. There are some aspects I like, and some things I don't. I really like how you used linest in your spreadsheet, especially using it to fit 2D data. Very classy! The only problem with using linest is that if the cells are empty, linest will return an error. Our previous spreadsheet (i.e. the one without air mass correction) used Linest, but we had several complaints so we switched to using slope and intercept (as they don't require all data cells to be filled in). I suppose we could thrown in a conditional statement like IF(ISNUMBER(cells)) or perhaps apply a filter to the data, but that just makes things ugly for people trying to figure stuff out. I suspect we wil have similar complaints, therefore we'll need to figure this out before switching to linest. We also can't set values to zero as that will skew the results from linest. Ilike how you included some estimate of error in your calculations. A while back I pitched the tutorial "levels" to the team. At the moment we just encourage our participants to calculate the standard deviation of the values resulting from their frames. The final tutorial will be all about uncertainty analysis and implementing things on your own. Linest will be particuarly helpful here as it can automatically compute the error in the fit parameters. I used these on the values included in the spreadsheet I posted a while back and the fits were horrible. I'm not sure if this is due to an inadequate number of stars being included in the fit or the data begin"just that bad". We'll have to investigate this as a team before we proceed with any inclusion of uncertainty in our spreadsheets. I like the idea of including the air mass calculations via. the altitude. That's an easy way of figuring it out. It also looks like you're using Young's approximation for air mass which is very good. Tom and Heinz both requested built-in computations of air mass. I think this is a good idea. The traditional method (i.e. using (RA,DEC) plus (Lat,Lon) plus the time of observation is more complicated for us, but it will require much less work for our participants. If implemented correctly they will only need to type in the time of observation after they setup the sheet for the first time. Here are the features from your sheet I think we should integrate: 1a) Using linest to calculate the coefficients and zero point offset instead of explicitly using the least-squares regression. We'll need to ask the team about this though due to the previous problems with linest 2a) Integration of Air mass corrections. Personally I'd rather use an outside calculator, but the team wants to do otherwise. In either case I want to use the coordinates rather than have our participants look up values out of a planetarium program due to both precision and ease of use. And to the tutorial I think we should add this 1b) Show the user how to modify the spreadsheet to include more/less comparision stars (really just modifying the arguments to linest so it's easy). This should take care of #1 above and give the user the ability to use as many comparision stars as they wish without needing to modify several other spreadsheets. I've aready done 1a and will modify the tutorial text (which I wrote about one-day after I posted the spreadsheet here) to include 1b above. 2a will take me some time to implement and we may just need to leave it to later as eps Aur is becoming visable again at more reasonable hours. I'll just suggest a few calculators until we can get this implemented. What do you all think, is this an acceptable compromise? Clear Skies, Brian
Hi Brian I took your sheet "Reduction-Intermediate_1.xls" and Jean Meus book Astronomical Algorithms and put them toghter so nowwe have the whole altitude/airmass calculation only dependent on date/time and observer location. On sheet 2 there is calculation of JD, local siderial time, local hour angle, RA/Decl of date, altitude and air mass. I tested with my original data from 2010-05-10 and the air mass was only 0.01 in difference between my original calculation and the new one, although the formula for air mass is not the same and I got altitude from a planetarium program originally. Thedifferences for Tc and k' was a little bit higher. This is depending on that I have used V and B-V from Tycho2 and you from AAVSO and the number of compairsion stars is not so many in your sheet. The end result for epsilon in my original calculation was 3.77 and I think we now got 3.81 Heinz, I don't think it's neccesarry to to a correction for refraction. The effect is quite small a bit from the horizon and if for example you enter the time 1-2 minutes wrong I think you got an bigger error. I think the most important is the relative diffrence in air mass between the stars.
You beat me to it :-) Yeah, that's the way to do it, very user friedly to have only to input coordinates and time data. CS Heinz
Sounds like a plan to me. As for the airmass calculation, all that is left now is the calculation of the altitude, depending on time, ground coordinates and sky coordinates. One question, tho: the altitude...should this be the "true" altitude or the "apparent" altitude (influenced by atmospheric refraction)? I know that some of the extinction methods take refraction into account and some don't which obviously is more important at high airmass only. CS Heinz
Hi Brian et al.-- I've been busy with other projects, and just got back to this one. Attached are two files. The first is a text format file output from AIP4Win's Magnitude Measurement Tool giving the raw instrumental magnitude, shot noise uncertainty, and airmass for each of the stars I selected in this image. Note that the line length is very long, so the lines will probably wrap in your text editor. The second file is simply the text file imported into Excel. Same thing as the text file. I added a label at the top of each airmass column to make it easier to spot them. To derive an extinction coefficient for the tricolor green of a DSLR, you would take a series of pictures over the course of an hour to get a wide range of airmass, then plot raw instrumental magnitude against airmass. The slope of the line is the extinction coefficient. The stars should all have the same extinction coefficient, and the value should be approximately that expected for V-band photometry. --Richard
Hello Heinz-- If you join the AIP4Win-Photometry YahooGroup, I post an announcement when each new beta becomes available. The way it works is that you must have an installed and registered copy of AIP4Win running on your computer. The beta versions consist of an AIP4WinV2.3.xx.EXE file in a Zip archive. You place the beta EXE in the same folder as the AIP4WIN.EXE in your installed copy, and then create a new shortcut to the beta copy. This allows you to run either the current release or the current beta. As each new beta comes out, I expect you to delete the old beta. Two key changes make photometry much easier with DSLRs. The first is: In Preferences > DSLR Conversion Settings, select BILIN interpolation. (Bilinear interpolation is linear; almost all other conversions are non-linear.) Also, select the DeBayer, Convert Color to Grayscale option, and set the Red Scale =0.0, Green Scale = 1.0, and Blue Scale = 0.0. Click Save to make this the default option. When you open a DSLR image, you will get only the Green color channel. To obtain this output, on the Report tab, select Raw Instrumental Mags, select the One Image and the Each Image... options. These settings give you output like that I posted a couple messages earlier. Please note that this is beta software. By definition, it is not finished, it may have bugs, and you must double-check your results. If you encounter errors or bugs, I expect you to report them to the AIP4Win-Photometry list so that other beta testers will be alerted to the problem and so that I can fix it in the next release. Repeat: As each new beta comes out, delete the old beta. I do not want new reports about old bugs in superceded beta versions. The user interface in all versions higher than v2.3.1 differs from that in v2.3.1. I changed the file formats to allow up to 99 filters and 999 comp stars and have made numerous other changes both small and large. There is currently no written documentation. However, if you're familiar with v2.3.0 or v2.3.1, you ought to be able to figure out how to use v2.3.28 and higher. You will need to create and save observer information, telescope and camera information, aperture settings, and an information file (*.STAR) containing comp star information. All of this information is used in generating the output reports. --Richard PS I have attached the EpsAur*.STAR file I am using with v2.3.28, with the name EpsAur*.txt. Place this file in the AIP4WINDATA folder and change the txt extension to STAR.
Richard, Very, very cool. I'm glad to see you're willing to accommodate the DSLR folks in your software, it's surely top-notch software! We'll be sure to point this out in our analysis tutorials (and revise our reduction documentation), but for those using IRIS or MaximDL, we'll still need a general solution. I've been reading through Arne's book and am fairly sure (read, "untested") that one doesn't need to follow a single star to get the air mass IF (1) they have at least three stars (a bare minimum) at slightly different air masses, and (2) the target star is between the calibrators (so we do interpolation, not extrapolation). This idea can be recognized from the following equation: (V - v) = -k' X + e(B-V) + d Where (V - v) is the difference between catalog and observed v-magnitude, k' is the atmospheric correction, X is the airmass, e is the transformation coefficient, (B-V) is the B-V mag, and d is the zero-point offset. Solve this equation using linear algebra and three calibration stars and vola, you've got all of the unknowns (k', e, d)! I think we might need to involve a color-dependent air mass correction term, k'', that means we'll need four calibration stars at a minimum and we'll derive the answer from a hyperplane using linear algebra. The bad thing is that the direct method of solving this doesn't let us use the additional calibration stars in the field of view, thereby letting the three/four stars give an absolute vote. I've got to derive the least squares fit for the equation of a plane given n data points... more fun stuff. After that, I think Excel will give us everything we need (basically taking determinants of matrices). I'll let you all know how this goes. All of this information will make it into the "Advanced" reduction tutorials in which we'll just present the theory behind DSLR reduction and let our participants do the implementation. A little fun for programming and math nerds like myself. Brian
fp55fI have tested solving for the k" (that is a correction term k" * AM * (B-V)). My experince is that the value of this coefficient is to small to be correct solved. In my testing the normal scatter between the stars when using a DSLR camera is so large that you don't get a meaningful value for this coefficient. I would suggest that it is better to use a fixed value for k" than solving for it. I havehowever seen the effect of k" in the changing value of Tc. See the following graph www.orbisol.se/bilder/amkoff.jpg. On the X-axis is airmass and on the Y-axis is Tc. As the airmass goes up (lower altitude), the Tc goes down (redder stars). Quite cool I think! Am I right in thinking that k" is the sloope of this graph? In that case it should be about 0.013
Hi!
Richard Berry & James Burnell have this to say about k'' in "Handbook of Astronomical Image Processing" (2nd ed. ): k'' is "small and hard to messure" and "many photometrists simply assume a value of 0.03" (chapter 10.3.2, page 290)
So this should give an idea about the value.
CS Heinz
Hi all! I found this page rather interesting: www. hposoft. com /Astro/White%20Papers/Airmass.html (Sorry, I had to put it on multiple lines or else the spam filter would reject my post outright (not even a chance to complete a CAPTCHA) (and Yes, we all know the author
). I'd think that this makes an excellent receipe for an Excel implementation of air mass calculation itself, for those who don't have AIP4WIN or comparable "integrated" airmass calculators. What do you think ? CS HB P.S.: Will try AIP4Win beta next
Heinz, Yes, Jeff's calculator will work fine for calculating air mass as will the AAVSOair mass calculator. I don't know what AAVSO uses under the hood right off hand (but I recall that they did discuss it on their page). Jeff's equations seem to be okay. The above spreadsheet takes the air mass values provided through one of those means and then calculates the zero point offset, transformation coefficient, and the extinction. I'd rather not integrate a macro for calculating air mass into the spreadsheet (as Iknow of at least three ways to do it, some of which get quite complicated and we also need to calculate the zenith angle... this will require a macro embedded in the sheet), atlhough if the group thinks it's necessary we can do it. Good catch though. Jeff has lots of good stuff on his website. Brian
Hi! Actually I'm having problems using the AAVSO calculator, I must be doing something wrong.... I get this result page (which also lests the input) http://www.aavso.org/cgi-bin/airmass.pl?jd=2010%2F05%2F28%2F21%2F25%2F00... but which is obviously wrong: the airmass is constant over many hours...and the star is eps aur, not Polaris :-) CS HB
Heinz, Indeed, I get the same result. I've submitted this as a bug to Aaron. Thanks for catching it! Brian P.S. The air mass value appears to be correct for the observation time specified, just not for anything thereafter.
Hi!
Not really sure, eps Aur was VERYVERY low at that time. I tried 3 different airmass calculators and got 3 different answers ranging from 1.something over 3.something up to 7.something (!!):-). I know that the different approximation methods diverge quite a bit near the horizon, but this level of disagreement was astonishing. The instrumental magnitude of eps aur and eta aur did indeed show a rather steep increase as the stars came closer to the horizon, but at a rather unbelievable rate. Zeta aur stayed more or less the same. So I guess I had some other systemeatic error involved maybe. I'll recheck with the latest beta aof AIP4Win during the weekend. CS HB
Heinz, An update on the AAVSO calculator. A module isn't working correctly after the big server move, Aaron said he might be able to fix it over the weekend. Until then, don't pay attention to the extrapolate air masses. You are correct, there are several different methods for calculating air mass. I've pasted a few methods of doing air mass corrections. Some are derived for plane-parallel atmospheres (i.e. parallel slabs), but most are fits to observed data. In all of the cases below "z" is the true zenith angle. A few of my favorites are:
The Secant Method:
Citation: I'm not sure where this comes from: Good to: This isn't really a good way of approximating air mass as it deviates fairly quickly. M(z)= secant(z)
Young's Simple Method
Citation: Young's "Observational technique and data reduction" in Astrophysics V. 12 1974 pg. 152. Good to:0.001 up to four air masses M(z) = sec(z) * [1-0.0012(sec(z)**2 - 1)]
Young's Complex Method:
Citation: Air mass and refraction" 1994. Good to: 0.001 air masses up to six air masses M(z) = (a * x^2 + b*x + c) / (x^3 + d * x^2 + e * x + f) where x = cos(z), a = 1.002432, b = 0.148386, c = 0.0096467, d = 0.149864, e = 0.0102963, f = 0.000303978
The Hardie Method:
Citation: Hardie, 1962 published in "Astronomical techniques." Hiltner, William Albert editor, Chicago, University Press (1962) pg. 180 Good to:This technique is a polynomial approximation to the data presented by Bempoard and is considered accurate to 0.1% to air masses of ~6.8 and 1% to air masses of ~10. M(z) = sec(z) - 0.0018167 * [sec(z) - 1] - 0.002875 * [sec(z) - 1]^2 - 0.0008083 * [sec(z) -1]^3