Teams / DSLR Documentation and Reduction / New Spredsheet version with optional extinction correction

New Spredsheet version with optional extinction correction

Citizen Sky is now officially permanent part of the AAVSO. In the coming weeks we will be moving additional content to the AAVSO site and freezing this site as an archive of the 1st three years of the project. Please visit the new landing page for future updates.

Hi all

Since the old thread was somewhat framgemented I thought I'd start a new one.
For most observers eps aur will now be observable at relatively low air mass, so I thought it might be a good idea to extend the Excel sheet with a simple "toggle switch" that will switch on/off air mass correction. In the attached file, you can do just that in cell F51

1 ==> air mass correction is performed
0 ==> air mass correction is not performed (= extinction coefficient is set to 0. No need to input observation location coordinates and observation time in this case)

Note that for both cases, the error in the comparison star mag fit is shown, so that you can actually see by trying both settings which method gives the better fit.

I'm still working on my backlog of data starting in August (!) when airmass was still ~2 for me, but in theory, with eps aur rising higher, the linear regression with air mass correction should become more and more unstable so that eventually the analysis w/o airmass should give a better fit.

Maybe someone can review the new sheet.

Clear Skies
Heinz-Bernd

EDIT: Files updated, see message below

EDIT: Files updated again, see message below

Comparing Spreadsheets

Hello again HB,

Just for fun I thought I would compare your spreadsheet (using "0", no air mass correction with Brian’s spreadsheet. The data I used had eps Aur almost at zenith (air mass = 1.0145). I was getting close but not exact results between the two systems until I realized some tabulated values for V-Cat and (B-V) Cat were slightly different between the spreadsheets. Once I standardized all the tabulated data the V magnitudes came out the same.

I did notice one difference between the two spreadsheets, however. HB, the formula you use for D Cat is D Cat = V Cat – I Mag. Brian’s formula is D Cat = I Mag – V Cat. Obviously, this makes no difference in the final results but I thought it was worth a mention.

Also, HB, is it possible to include the graph of (B-V) Cat vs D Cat in your spreadsheet? For those of us who are more visual than analytical it really helps.

Thanks and warm regards, Tom

Thanks for the checks

That adds quite some confidence. Definitely, a Plot would be a good visual check. I thought about two other extensions: 1) Some automatic quality checks, that, if violated, would make some "traffic lights stype" indicator cells go red and result in a warning test to be displayed, e.g. when the coeffiecents resulting from regression have unusual or completely implausible values. 2) an optional 2nd order extinction coefficient that is not calculated by regression ( I think that would only make the calculation quite unstable) but would default to some fixed value. We discussed this here before. For example, in "Handbook of AIP", it's proposed to used a fixed value of 0.03. Clear Skies HB

Spreadsheet Enhancements

These sound like excellent (and sophisticated) enhancements. I just hope they'll run on my older ('97) version of Excel. If not, well, this old timer has needed a reason to finally upgrade. 8-) Warm regards, Tom

oops

I just discovered a liitle bug in my sheet: the (only informative) check star magnitude calculation should also depend on the "toggle". I'll add that tomorrow. This won't influence the target star magnitude figure.# CU HB

New version

Hi! I've uploaded a new version of the sheet as attachment to the original post. Changes:

  • All internal tables are now on a separate sheet. Looks cleaner IMHO
  • 2nd sheet is now reserved for a How-To (will come later)
  • Changed color scheme: User input is now expected in AAVSO-website-blue fields... I needed yellow for ...
  • Data Quality indicators: For now only two: fit errors for comp stars and check star. More to come (e.g. airmass value plausibility)
  • Last but not least: two plots: plotted are the correction terms for transmission coefficient and extinction coefficient vs (B-V) and airmass, respectively. Can also be used to judge if enabling airmass correction did improve the fit.

Tom, do those diagrams do what you wanted? CS Heinz-Bernd

New Version

Hello HB, I really like the "New Version". The graphs are great. Thanks, too, for correcting my mistake with my sign for longitude. I've attached my results for a 1 Nov observation (V mag 3.792 for Eps Aur). I noticed in another post you were headed out to observe at about that time. Are our results compatible? Warm regards, Tom p.s. In row 13, what does "LST" represent? I thought it was Local Standard Time at first but the numbers don't seem to match up with that interpretation.

AttachmentSize
eps mag 2010-11-01.xls 39.5 KB
Very Reasonable V-mag

Hi Tom, Very reasonable V-mag. Dr. Bob's digikam (a JPEG device) got 3.76 (Nov. 1/2). Jeff Hopkins got V= 3.7428 +- 0.0034 (that's a standard deviation, not uncertainty) on Oct. 30/31 so you're surely in the ballpark. If you look at the errors in the fit zeta, lambda, and rho Aur don't fit very well. As for "LST" in row 13, it's a measurement of time called theLocal Sidereal Timewhich is required to calculate the air mass. It's fixed to background stars and results in a day that's about 23 hours 56 minutes long instead of 24 hours even. Something is also broken in the spreadsheet you attached. The V-mag isn't updating when you toggle the air mass correction on or off. A couple other notes (mostly for me and Heinz):

  • We shouldn't put the Imags for the check star and target star up at the top. We use the check star on a per-frame basis and the target object similarly, therefore their imags should be extracted one at a time and included below.
  • I think we should move the Fit Value/Error table back to the Data sheet. The information contained there is very useful (for instance it can show that you measured the wrong star which resulted in a bad fit) and shouldn't be hidden on a different sheet. The other table can stay, but they need to be labeled.
  • After we have the final version complete (which I hope this spreadsheet almost is), we need to propagate the errors from the fit. I would also like the errors from the calibration stars to be included... I'll think of an easy way for this to be done.

Cheers, Brian

V-mag results

Hi Brian, Thanks very much for the comparison data. I'm encouraged. It looks like you and HB are rapidly closing in on the final version of the Air Mass Spreadsheet. I congratulate you both on this impressive accomplishment. Tom

Hi Brian, >Something is also

Hi Brian, >Something is also broken in the spreadsheet you attached. >The V-mag isn't updating when you toggle the air mass correction on or off Actually it does, you just have to format the cell in question to show more significant digits :-). I think this is really good: it shows that at an airmass this low, extiction doesn't have a big effect anymore, but if you nevertheless try to correct for it, it doesn't mess up your results. >We shouldn't put the Imags for the check star and target star up at the top. We use the check star on a >per-frame basis and the target object similarly, therefore their imags should be extracted one at a time and >included below. You mean the I mags should be near the result table, as for the optional additional images? Yeah, I agree. >I think we should move the Fit Value/Error table back to the Data sheet. Ok, I think logically they would belong in the "data quality" section. Ok ? > The other table can stay, but they need to be labeled. OK. I'll see when I've got time to do all this, but I think we are close to finished now. CU HB

Hi Heinz-Bernd, I really like

Hi Heinz-Bernd, I really like the new spreadsheet. Moving the intermediate tables to a new spreadsheet does make it look nicer, but I think having that information right there is handy. If we are going to move them we need to make sure the tables are properly labeled. The residual plots are a very nice enhancement, but I'm confused by them. The vs. air mass residual plot doesn't subtract the air mass effect, neither does the vs. color plot. For these plots to be called residuals they really need to account for the mentioned effect instead of just displaying it. In the attached spreadsheet I just plotted up the "error after fit" vs. airmass and "error after fit" vs. color and then looked at the resulting slope and r-squared coefficient. For the vs. color diagram the slope is nearly 0 and the r-squared is 0.31, implying little-to-no correlation so we've solved that problem. The vs. Air mass plot has a slope of 0.04 and an r-squared of 0.71 implying there are second-order effects that need to be considered (which we expect to be present as we don't account for second-order extinction effects). I would suggest we display these two plots instead. I should also point out that the slopes of these lines also show up (in magnitude) in the error cells in the Planar fit table on the data sheet. The data quality indicators are a good idea. We should revise our introduction tutorials to conform to the new coloring scheme you implemented (I'll take care of that once we post up the spreadsheet). I think we should keep the instructions for how to use the sheet on the main website for one major reason: we can fix errors and provide updates. The secondary reason is that people can comment on the page where we post the instructions. I think this helps promote dialog between us and the users of our software. What do you all think? Do we think this is ready to go main-stream?

AttachmentSize
2010-10-31-Reduction-Intermediate_mult_airmass.ods 35.27 KB
Hi Brian, Just to explain,

Hi Brian, Just to explain, what I tried to do with the digrams was to show the color and extinction correction separately (as it's difficult to do that in 3D), so that the slopes in the diagrams would indicate the transmission and extinction coefficients, respectively. Without extinction correction, the first diagram would actually be the same diagram we had in earlier spread sheet versions (color vs (instr mag - catalog mag) ). So the diagrams would show the first order effects, not a remaining second order effect, I guess the "residual" in the caption was quite misleading. But I'm ok with the alternative diagrams, as with your other suggestions. CS HB P.S.: I have to run now, it's a clear night :-)

Hello again Heinz, Ah, okay.

Hello again Heinz, Ah, okay. For a few minutes I thought that's what you were trying to do, but as you mention the "Residual" thing in the caption was indeed misleading. If you re-label your plots and/or put in the residual plots I suggest it should be more than adequate. I hope your clear night is going well! Cheers, Brian

next version

Hi! So I did the following changes:

  • I moved back the most helper tables related to extinction calculation to the main sheet. This is more than cosmetic: it will allow users to add/remove comparison stars by just inserting a line in the comparision star table and filling in the catalog data, all the helper table formulae should then be auto-created automatically. That would not happen if they were on a separate sheet I guess.
  • I added a new plot for residual magnitude versus air-mass times color index, to estimate second order effects. Not sure this makes sense.
  • added a plausibility check for the air mass data ranges. I assumed values > 10 indictate an error. I guess most typing errors that result in significant errors in observer site coordinates (e.g. switching signs) or observation time (e.g. local time instead of UTC) stand a good chance to be discovered this way
  • Instrument magnitudes for check and target stars are now entered in a line of the result table.
  • moved the LST calculation to the "Internal" sheet because it doesn't seem to be important enough to be displayed to the user.
  • Added a hint for the correct sign of geo-coordinates of observation site
  • Removed the Doku sheet

I also moved around some cells for aestethic reasons, hope that didn't break anything :-) CU HB

AttachmentSize
2010-11-07-Reduction-Intermediate_mult_airmass_0.ods 46.2 KB
one more update ...

yet another helper table went back to the main sheet CU HB

AttachmentSize
2010-11-07-Reduction-Intermediate_mult_airmass_0.ods 46.07 KB
2010-11-07-Reduction-Intermediate_mult_airmass_0.xls 41 KB
Looks good

Hi Heinz, This looks good. I'll re-write the documentation today and get it posted on/before Monday.. then we can get a blog post out about it at the beginning of the week too! Brian

I like it!

Hello Heinz,

I gave your new spreadsheet a try and found it performed very well. The "toggle" idea is neat! I’ve attached a copy of my results using data from 30 April 2010. The air mass was 1.97. The average error with air mass correction (1) was 0.0189 and without correction (0) was 0.0451.

Eps Aur magnitude was 3.737 with correction and 3.776 without correction. I compared these results with the Hopkins Campaign’s V magnitude light curve (http://www.hposoft.com/Plots09/VBand.JPG

). The range of magnitudes for30 April is between 3.75 and 3.78. The reason these results are closest to my "without correction" magnitude may be that the majority of Hopkins participants are not performing air mass corrections.

Warm regards, Tom

AttachmentSize
HBSheet 2010-04-30.xls 26.5 KB
sign of coordinates

Hi Tom, I had a look at your spreadsheet, it gives much better results if you use a negative sign for your observer location longitude. Of course this is pure convention, or is there an accepted standard in astronomy whether W or E should be negative? (I would not be surprised if there were two opinions on that, one "American", and one "English") :-) I'll include a hint in the next and some checks in the next version of the sheet. CS HB

Greetings HB, East longitudes

Greetings HB, East longitudes are positive, West are negative. In conversation we tend to say 107 degrees West whereas we really mean -107 degrees East. Brian

Greetings HB, Very good idea!

Greetings HB, Very good idea! I'll use this as the default air mass calculator when I get around to posting content online (Tom Pearson and I have been revising the text in the tutorials in preparation for an upcoming article). Would you like to write a how-to tutorial for this spreadsheet? It could be similar in spirit to the non air-mass corrected one. Brian

Sure, I can do that

Sure, I can do that How-To. CS HB

Powered by Drupal