How to test digital camera photometry precision?

Hello WolfgangI am also using a Canon Powershot except it's a more recent G9. At beginning I did analyse my images using IRIS manually and next I did develop my own software. Results seems to me very coherent at 0.01 level or better even if the smaller sensor of the G9 generate more noise than the APS format of DSLR. My reports for epsilon and zeta are well in the average of the AAVSO light curve. I typically make a series of 12 raws with 4 seconds exposure at 100 ISO and F 4.8 using the large teleconverter that results in 400 mm focal length(24x36 equivalent) This provides a 5 degrees field and some star trail that avoid saturation when associated with some defocusing (setting 5 meters distance instead infinity) That field give me access to six bright stars I use for my test: eta, zeta, epsilon, HR1558, HR1550, HR1615, plus some others depending sky conditions.So far I have tested the accuracy of my process comparing my results with the reference V magnitudes of the five stars (other than epsilon) covering a significant color index range, and also calculating the standard deviation from my series. My software also includes an evaluation of the read noise in the picture and other parameters that are good indications on how to optimize the exposure conditions. I think this is most of what we can do. I also made some more analysis of the Powershot itself: its spectral response, offset, linearity of the sensor, sensitivity of the R G G B channels... It's usefull to understand the saturation conditions and also how to achieve the best color correction to V Johnson.My software uses "dcraw" from Dave Coffin to decypher the Canon CR2 format to a "pure raw" 16 bits Bayer format without any interpolation or rescalling. I next works on the four R G G B plans for my data reduction. I work under an APL system I consider a superb tool to analyse data and make experiment with. But APL systems are not common by present times and probably very few are using it for astro works ! If some APL's guy is interested I would be very pleased to share views. I am now considering to make experiments with simpler DSCs than the Canon G series (present G models have functions equivalent or even better than DSLR !) one issue would be the focus and the next jpeg... I will probably test a Nikon P5000 I have access to. If you are interested we can discuss and exchange data on that various aspects.Regards,Roger

Hello Roger,thank you for your comments and pointers!At the moment I take my images at shortest focal length of 7mm at f/2.0, 15 seconds exposure and ISO 400 setting. I try to get a linearity assessment for the camera and measure some other stars as well.I am quite far away from 0.01mag precision as it seems but still much better than I can do visually.Please see accompanying the light curves of Epsilon Aurigae (green image) and of Delta Cephei (also green image) where not JD but phase is plotted on the x axis.Best regardsWolfgang

Hello Wolfgang,Sorry for that late answer, I was on trip and didn't access CS. I saw your light curve and I am impressed with your delta-cep results, very nice. I was planning to also do it but had no time up to now. I put my own results on your eps-AUR chart (hereafter attached) and I think we have a good correlation (except your three first points) I recently made an experience with a DSLR ( Nikon D70s) but got more problems than with the Powershot. The DSLR has a much larger imager (APS 23.4 x 15.6 mmm instead 8 X 6 mm for the G9) that results in much lower noise but at end there are systematic errors I have not been able to fully eliminate. It seems that comes from filters having a more complex response, difficult to compensate by software, and also some non linearity at bottom of the light curve (not a saturation). I understand you use a much shorter focal length than I do and it seems most people are also using relatively short focal length from what I see in others notes. I also realize that most and you use much stronger exposure than I do, that's: 88.8 mm true focal length (8x6 mm sensor) F/4.8, 4 seconds exposure at 100 ISO, at that level I get about 2500 ADUs at the brightest point that's ok with regards to saturation risks (full range is 4095) I could push a little bit more. At first impression your 7 mm focal length at F/2 and 15 seconds at 400 ISO seems a lot: 2.5 stops x 3.75 time x 4 ISO = 37.5 times mine... in fact it's not true in case of point source like stars, the photons capture is not a function of F itself but aperture diameter (surface) that's in my case 88.8 / 4.8= 18.5 mm against 7 / 2 = 3.5 mm in your case, that's an aperture surface ratio of 28 ! Indeed we do about the same overall exposure ! The only issue I see is that you should use a sensitivity of 400 ISO that's not good for noise. Getting more ISO in our digital cameras means simply pushing the gain of the read amplifier, that also increases the noise from its input by the same amount (said "read noise"). That also means using less electrons in the same ratio resulting in less dynamics (range) and more noise from quantum fluctuation (that's square root of n, and our n is so small in 8x6 mm sensors, probably less than 1000 per pixel!!). From what I see on real data the dark currents noise is insignificant with our very small pixels, most comes from read noise and fluctuations. By the way dark process is not so important, it could even add read noise with negative effect ! What do you think about using a much longer focal length ? Best regards,Roger

"how to plot the "TG" (tricolor green) measures I reported to AAVSO in the light curve generator?"Check out the VStar tool that we are developing: http://www.citizensky.org/teams/vstar-software-developmentClick on Download VStar Now. From the File menu choose New Star from AAVSO Database and select epsilon Aurigae. Then select Change Series to pick which filter bands to plot. There is an option for tricolor green.

Hi all,I've also been looking at the possibility of using my Hutech modified DSLR for photometry. So decided to look at one of my older "pretty" images that I took about a month ago. The image was of M31 through a 70mm refractor, and I wanted to examine some of the stars in the field and compare the published magnitudes with the value I get from a software measurement of the intensity (either IRIS or Mira)I have about 15 light frames, which I reprocessed with dark frames, stacked, and split out the green channel. I do not have flat frames. I then opened up the resulting image in IRIS and looked for stars where the peak was less than about 2800 ADU (4096 max for the camera). I had IRIS calculate the magnitude of a group of stars and then compared with the listed magnitudes of each star in Starry Night. Here are the results: IRIS Listed Mag mag Offset ============================ -10.48 11.15 -21.63 -10.94 10.78 -21.72 -10.72 10.96 -21.68 -10.06 11.68 -21.74 -9.6 12.05 -21.65 -9.34 13.1 -22.44 -9.18 13.2 -22.38 -8.81 13.35 -22.16 -8.69 13.75 -22.44 -7.78 14.4 -22.18 I did not apply a color transformation.Even though I get an R-squared of 0.98, I am still concerned that my response isn't completely linear. The offset of the measured magnitudes with the published magnitudes seems fairly consistent for stars "in the middle range" (roughly they peak between 30% and 60% of capacity. When I look at the dimmer stars, they seem to start to diverge. If it matters, the background intensity for all of the stars was around 290 ADUs or about 7% of capacity. I'd like to know if this is expected, or maybe am I doing something wrong? Next week I will do some more testing, such as verifying that the response is truly linear from 0-60%, and also calculating the Transformation Coefficient. I will also try taking images of Epsilon Aurigae and see how my results compare with others.Any feedback would be appreciated. Thanks.-Mike DurkinDMPA

Hello Mike,First I would say you could use the "Magnitude Constant" of IRIS to eliminate the offset of your reference star. Next your background being at 290 you should not have problem of clipped black level and/or clipped noise at black level (that's an issue on some cameras... like Nikon ! ) Apparently you didn't use IRIS for decoding the files from your camera. What kind of file is it ? sort of raw ? How did you decode them ? Depending the case the software being used could apply or not a gamma, makes filtering, interpolation, that affects the fainter star... various tricks like D-lighting ! Your results seem coherent for the brighter stars (10.6 - 12.05) within 0.1 that's ok under such conditions. Next as soon as 12.3 most stars jumps by -0.8, brighter than listed values. I doubt it would come from a linearity issue, our cameras are not so bad on such short range ! I also doubt the color index alone would generate such difference. I would check where the stars are located on the picture as it could be a flat issue, factor like 0.8 magnitude could exist between the center of the field and the side of it. I had a similar case with a tele lens on a Nikon DSLR. I made a mapping of the errors then the flat issue was obvious. The needed correction was more than a factor 2 at the corners of the picture !Another aspect is that your range is large, more than 3 magnitues, that's not small compared to the limited dynamics of digital cameras. Next your background is very high (290, I suppose offset being eliminated ?) against the level of your dimmer stars (should be 2800/~20 = 140 or less ) Under such conditions it's probably difficult to reach a good accuracy due to noise of various sources (high background fluctuation, dark read noise, signal fluctuation...Yours truly,Roger

Roger,Thanks for the reply.You're correct that I didn't use the "Magnitude constant" in IRIS. The files were initially Canon RAW format. I used MaximDSLR to subtract the dark frames, convert to RGB, align, and stack. After that, I split into RGB, and saved the green channel as a FITS file. Then I imported the FITS file into IRIS. I did not apply any sort of stretching or curves.All of the stars are from the same 2 portions of the image (top-center or bottom-center). I couldn't use stars in the middle because that's where M31 was in the image. I also stayed away from the left and right edges of the image because I can see that I need both a field flattener and flat frames to correct that part of the image. I don't THINK that my discrepancies are a flat issue since the variation of the background intensity was very small (within 6 ADUs), but your suggestion of mapping the deviation is a good one.This image was not originally done for photometry, which is probably why my background was a little high. I was following Jerry Lodriguss's rule of thumb to have the histogram "mountain" about 20-40% of capacity. For Photometry I assume that you expose a little less than that, but at least long enough to make sure the "mountain" is sufficiently high enough so that there isn't any clipping on the black end. I would also defocus a little to spread the starlight out and keep the brighter stars from reaching the non-linear part of the pixel response.I didn't initially think that my magnitude range was too large, I just thought that I needed to stay in the linear response range. Now that I plug in some numbers using your background guideline and an upper limit of about 2450 ADUs(60% of 4096), I come up with a range of about 3 magnitudes.I obviously need to test my camera more, and specifically for photometry purposes. At least I think that it is possible with my equipment, just under certain constraints.-Mike Durkin

The thing I would check first is the Starry Night values. My guess is that up to the mag 12.05 star, the values are coming from the Tycho photometric catalog, and those fainter are coming from one of the photographic catalogs. These latter values can easily have a magnitude offset and should not be considered "standards" in any way, shape or form. So don't blame your camera yet! As you go fainter, there will also be increased photometric uncertainty just caused by lack of photons from the object, so the calculated offset will tend to vary more for these stars. The way you get around that is to measure lots of them - there will be a cloud of such points on a graph if you plot camera magnitude vs. catalog magnitude, but the least squares line should pass through the median values.The main catalogs of photometry for m31, such as Phil Massey's great work (seehttp://www.lowell.edu/users/massey/lgsurvey.htmlfor references) typically look at faint stars, so things in the 13-15mag range are saturated. You can use stars from the TASS survey (available through VizieR: http://vizier.u-strasbg.fr/viz-bin/VizieR) for crude estimates of brightness (but better than the photographic surveys!). Our own APASS survey at the AAVSO (see http://www.aavso.org/news/apass.shtml) won't cover the M31 region until January.So my suggestion to you is to find reliable photometry for your comparison. Either find such values in the m31 field, or perhaps image a field along the Celestial Equator where lots of good photometry exists. Once you know that the reference photometry is good, then you can see whether your camera reproduces that photometry.Arne

Hello Mike, Yes, I think the best is you make another test on a better known field: why not "epsilon AUR" one ! We have good comparison in this area. Another point I didn't realize is the fact this image was well focused, by the way the photometry could be affected by the pixels separations and also the fact the signal integration involves only few pixels which means a total of few ADU's and, more important, few electrons being accumulated, resulting in a limited accuracy. I usualy defocus up to reaching a spot of about 15 pixels diameter with peak ADUs about 2500 (/4095 max) at the brightest pixel and 100 ISO (that to maximize the electrons count - not the ADUs only !) Under such conditions I cover a range from "eta AUR" (3.172) to stars about 7.5 in agreement with the V magnitude noted in the Tycho 2 Catalog (Tycho VT magnitude is sometimes different by 0-0.1 from that standard V magnitude) The typical delta is < or = 0.01down to mag 6.15 and next increases to 0.02 (sometimes 0.05) up to mag 7.5 At that point the total ADUs count per staris very low ( ~600)and the electrons count even smaller ( ~300) that resultsin SNR as low as 10~20... Yours truly, Roger

Arne, Roger,Thanks for the feedback.I wasn't sure how reliable I could take the magnitude and color measurements in Starry Night. I tried searching the documentation, so I can't tell exactly which catalog it is using.Before I try taking images of stars again for photometry, this past week I wanted to test out just how linear my DSLR is. I set up my camera pointing at a blank wall, and then took exposures of varying lengths (.005 to .5 sec). I'm attaching a spreadsheet which has my results, including a couple of graphs. Please let me know if anyone has problems reading the file.For both sets of data, I converted all the RAW images to 16 bit TIFFs using Canon's "RAW Image Task". The difference between the two is that for the first set, I checked the box that said "Linear Image". For the second set, I did not check that option. I noticed that the ADU range was rescaled from 0-4096 to 0-32768 when the files were converted to TIFFs.The X axis of the graphs is the exposure time in microseconds (column A of the spreadsheet). The Y axis is my measured average of 4 random data points of the green channel near the same pixel location on each image (columns F and M). It looks to me it seems that I can get a very linear response when I save the RAW files using the "Linear image" option. This included images where the histogram over 60% of capacity. When I do not use the "Linear Image" option, it looks like the linear response starts failing at values that are over 30% of capacity.So I guess my questions at this point are: - Is converting a Canon XT RAW file to a "linear image" TIFF file like I've described a valid way to obtain data? - Is my camera (Hutech modified Canon 350D) not considered accurate enough for photometry?Thanks,Mike Durkin

Happy New Year to all of you ! and superb light curves from eps AUR ! Mike,Your xls file is ok no problem.Your results seem to me very good, the small deviations in your series 1 is probably due to some exposure variation, I am not sure shutters are always at the exact displayed value...Series 2 is just the same thing but "gamma" corrected. Standard electronic imaging uses a non linear representation of the light levels, like: S = L power G , where G is in general 0.44 for video, the cinema uses 0.55. Video applies also some linear sections at low and high light levels. Next the TV or monitor displays, printers... convert it back to linear using the opposite law: L = S power 2.22 or so. This is applied since the old days of the TV in the 60s. It fits to the human vision (that has an incremetal sensitivity like: delta L / L) matching the noise sensitivity to it in analog systems and the digitalization steps in digital domain. Most of our DSCs use the sRGB standard that follows the 0.44 norm. But not all devices are very accurate applying gamma that makes it difficult to use for photometry. If you want to obtain the absolut "raw" values from the ADC of your DSLR, without any rescalling or offset elimination or gamma, you could use "dcraw" from Dave Coffin. In fact most softwares reading DSLR raw files are using this routine. The command: dcraw -D -W -g 1 0 -6 IMG_xxxx.CR2 will generate a 16 bits "pgm" file being a "pure raw" translation of the Bayer format of the sensor of your DSLR. Very interesting for our purpose.The command: dcraw -d -W -g 1 0 -6 IMG_xxxx.CR2will generate a 16 bits "ppm" file being an RGB image rescalled to 16 bits, linear.His site: http://cybercom.net/~dcoffin/dcraw/Yours truly,Roger

Hello everyone,I've been doing some more testing with my modified DSLR. This time I used a 50mm lens and took images of the area around Epsilon Auriga. I also applied dark frames and flats to each image. I stacked 10 images, each 5 seconds long. I also had the lens out of focus to spread out the light from each star.With my tests so far, most results are within 0.03 of the published magnitudes. Initially I did include Sigma Aurigae, but I think it is too red (B-V=1.42). When I plotted the measured magnitude against the color index of all of the other stars, they formed a fairly linear plot (R-squared=0.91). With Sigma Aurigae, R-squared is 0.83. I'm posting up a spreasheet of the data excluding Sigma Aur. Let me know if these results seem to make sense. I would also like to know what is the limit of the color index that should be considered for DSLR photometry.Thanks,-Mike Durkin

Hi, So I've done some more calculations on a set of images Itook of the area around Epsilon Aurigae on Jan. 31. Itook 60 light frames, as well as a set of dark and flat frames. Isplit the light frames up into groups of 10 and stacked, so Ihave a total of 6 processed and calibrated images. Iused magnitudes and color indexes of comparison stars taken from the AAVSO site and a spreadsheet in a post by Tom Pearson in order to calculate my Transformation Coefficient. When Iplug the numbers into my spreadsheet, and average everything out, Icame up with a TCof 0.081and an R-squared of 0.903. Icalculated Epsilon Aurigae's magnitude by comparing it's magnitude with 7 of the stars in the same field. When Iaveraged all of the magnitude calculations, Icome up with a value of 3.684 with a standard deviation of 0.016. The first thing I'd like to know is if my methods of calculating the Transformation Coefficient and the magnitude of Epsilon Aurigae are considered acceptable? Another thing Iam a little concerned about is that even when stacking 10 light frames, Iwas getting a variation in magnitude of some stars as much as 0.1. Is that considered normal, or shuold Iexpect a lot less variation? As far as Ican tell there weren't clouds in my images, but Isuppose it is possible that there could have been thin cirrus clouds that were causing some fluctuations. Inotice then Icompare my result with others at the same time (JD 2455228.68) that my measurement is similar, however it ranges from 0.02-0.05 brighter. I'm attaching a spreadsheet with my measurements. Any feedback or comments would be appreciated. -Mike Durkin

Hello Mike, 60 frames is a lot!I do not havesuch courage ! I usually use only a dozen with a DSLR (450D) and sometimes up to 24 with a DSC (G9) I do not stack but analyse each image, for me it's as fast as stacking and enables to make a good SD calculation (anyhow my software do it automatically).In term of accuracy stacking oraveraging is no difference. I dotnot use the AAVSO TC methode but a BSC "blue spectral compensation" to V Johnson. If I do theclassical TC calculation I usually get something like TC=0.14~0.15 (not 0.08) either for the DSLR or the DSC but as said hereabove I do not usually use that methode (only for comparison).Resulting TC magnitudes are in general similar to such from BSC, except BSC doesn't depends from preconceived B-V ! (sometimes erroneous !.. and sky conditions dependant...) Iattache a typical result on a set of 12 stars ranking from 3 to7 magnitudes (R100226.png) each line is the result ofa singleframe, the line before lastis the average, the last line is the SD. You can see that arange of 0.1 mag iscommon for 12 frames for fainter stars but not for an averaging of 10. A resultat 3.684 for eps AUR on JD5228 seemstome very good,my own is3.693 (to see R100131.pngattached) What I would suggest is to tryto optimize your shoting conditions to improve your SD with much less frames (increase exposure at low ISO).Fixedpattern noise processing is probably not useful (not in my conditions) maybe adding more random noise... only the lens vignetting correction is important. It's also better to keep the ADC offset (if existing on your DSLR) up to the photometry step. My present shoting (optinized) conditions with the 450D are: APS-c CMOSimager, 100 ISO, 6 sec., 200 mm 1:4 lens (50mm aperture) , defocus~100 pixelsand some star trail resulting in about 7000 ADUs max per pixel (14 bits). Total ADUs acumulatedin theeta AUR spot are~220000when sky is clear at zenith.For the G9 conditions aresimilar except the lens aperture is only 18 mm and the resulting SNR is not as good. Yours truly, Roger

Hi Mike, I had a more detailled look at the operations in your XLS, for me it's no problem. I think your camera has amodified IRcut filter, it's probably the reason for the TC=0.081 instead of the classical 0.14. That new filter enhances the red response, it's coherent with that 0.081 TC. As said this morning I only suggest that you try to increase strongly the photons accumulation, using low ISO (limitation by ADC saturation is proportional to ISO setting) longer exposure, larger lens aperture surface (longer focal lenght generally helps), defocus to 100~200 pixelsper star, possibly including star trail. Yours truly, Roger

Roger, You are correct that Ihave a modified DSLR, it is a Hutech modified Rebel XT. Ialso found magnitude and color values in the documentation on this site that had 3 significant digits rather than 2 for the standard stars. Using those values brought my TCvalue to .087 and my measurement of Epsilon Aur to 3.689 +/- 0.018, which is in good agreement with your measurement. Processing 60 (6 groups of 10) images wasn't too much of a pain. Iwould have MaximDSLR stack 10 light frames and also apply the light and flat fields. On a separate note, Iused tissue paper as a diffuser to cover my camera lense. The light pollution where Itook these images is bad enough (NYC)that Ican get a usuable flat frame with a 50mm lens and just a few seconds exposure. I'll try your suggestions. Iwas already defocused to about 50 pixels diameter, and Iwas wondering if that was too much, especially if the stars being measured were close to other stars. Imay need to figure out a way to do flat frames if it is necessary. My trick using tissue paper for flat frames won't work in my usual location where the sky is maybe a magnitude or so darker than NYC. Thanks so much for the feedback. -Mike Durkin