My much awaited *DUE* blog post is here fellas (FINALLY). I must say that it gives me grave sadness that this post is going to be only about jargons and I am going to have to curb my humour to do so (which I believe I have, okay?). If I don’t, then the length of this blog post might climb the everest and inevitably a 128 bit floating point data type (when available and supported) won’t be able to count, in periods of caesium 133, the reading time of this post. If you didn’t understand this, READ MY PREVIOUS BLOG POST (publicising my own blog like a pro). OH BTW, if and when 128 bit floating point data type is supported by FITS, remember, I need to be informed FIRST. It is a matter of life and death I must tell you. *FITS, why you testing my patience levels? STAHP*.
Getting back to the point, the last two weeks I have been quite ambidextrous and I have successfully dabbled in multiple works (GSoC is included there I think. Just kidding, I NEED to pass my evaluation). While there is a marked difference in my vocabulary (thanks to GRE), the work for these two weeks has been more about understanding (a lot of it) accompanied by my beloved tea, rather than actual coding (irony eh?). I will talk about it in the following brief paragraphs (Don’t be so naive, “brief” is just a euphemism).
I never really officially introduced my GSoC project here. So here it goes *workaholic aarya in action*, a part of the project abstract:
Astropy has emerged as an integration of several independent packages (PyFits, asciitable), each responsible for an individual task in the storage, reduction and analysis pipeline of the astronomical data sets. It has mostly unified the interface in order to make these entities coalesce. However, the need for Astropy to shine as a single package and not just a collection of functionally independent modules, calls for seamless interoperability between the three special and powerful constituents of Astropy : coordinates, time and units and the underlying Astropy Table writers. We seek to develop a protocol that allows their storage, the mixin columns, in FITS and ASCII ECSV, while still ensuring round-tripping.
Now the catch here is that, while I strive to achieve 100% round-tripping of the extensive metadata associated with the rich
~astropy.coordinates.SkyCoord through FITS, abiding by the rules set by the FITS standard and having an implementation which is compliant with it and still manages to preserve complete information in that FITS header with limitations of 8 bytes keyword name, 80 bytes keyword record and a well-defined approach to every possible use-case that astronomy could face, is a great pain in the, well, everything that a human body possesses.
It might seem like FITS is actually inflexible, sorry for that portrayal, when in fact it is just the opposite. FITS (Flexible Image Transport System) is the data format most widely used within astronomy for transporting, analysing, and archiving scientific data files. FITS is much more than just another image format (such as JPG or GIF) and is primarily designed to store scientific data sets consisting of multidimensional arrays (images) and 2-dimensional tables organised into rows and columns of information. To put together everything that an astronomer ever longed in his entire life and to do so beautifully and with such flexibility for each person’s taste is truly incredible (I do not want to cry over this). To get a super-quick overview of FITS you could go to https://fits.gsfc.nasa.gov/fits_primer.html. There you will also find detailed documentation for almost about EVERYTHING (just exaggerating).
I talked about time in my last post, remember anything except my alluring humour? (well thank you). I did so because my current focus is to allow using the existing (but somewhat new) FITS standard for storing Time objects. I remember Tom saying the following to me during the proposal writing time, “This alone could take much of the summer but would be a very important contribution. It will be somewhat exacting work as the standard is long and detailed (22 journal pages), and does not precisely map to the astropy Time object (though mostly it is a reasonable match).” (I now truly understand why. *sobbing has slowly escalated to screaming*)
And if you really intrigued by my discussion regarding time scales in my previous blog, you should definitely read http://www.iausofa.org/sofa_ts_c.pdf. Hands down the best explanation you will ever find.
So I hope we are on the same page. I have to round-trip astropy Time and the Journal is looooooong (yes we are 🙂 )
The reason I am not getting into too many details is that it would take me hours to do so and I already speak too much (duh, I think we have already established that).
But to talk about jargon as I promised (damn it), here is the work that I accomplished until the start of last week:
- I managed to allow
~astropy.units.Quantity to be written to and read from FITS files, with relevant metadata of course, before the official GSoC period started. I also achieved the same for VOtable (I also did solve a couple of issues here and there and added a few minor features. FYI, I sometimes, with no reason, look at the contributors list and feel a little (a lot) exuberant seeing my name in the list *not creepy at all*)
- Time was really interesting to read about so I did so willingly. I read about time scales, got familiar with the norms and abbreviations, understood and played around with astropy Time and then finally read the “Representations of Time Coordinates in FITS” JOURNAL *sigh, I had to*
- For a detailed look at my work, you could visit https://github.com/astropy/astropy/pull/6176 and try and understand what is going on there (If you have the patience, anything is truly possible. Except writing your own fits module 😉 For those who did, what did you eat? Patience for dinner?) I have managed to round-trip Time through FITS to quite an extent. I still need to incorporate certain features not directly supported by the FITS standard, for eg. vectorized location
Then the last week and a half were jam packed doing the following:
- Cleaning the code according to all the reviews
- Bringing together everything related to the FITS time coordinate frame in one place, the FITS_time class, in which case the knowledge of the names of the keywords could be kept in one place (My local tests run, eeeeppp)
- Rigorously testing and checking for unusual corner cases (I have created a separate FITS_time test class to do so)
- Reading the FITS journal again to skim through a few topics again to understand the possible use-cases and to find keywords which could possibly solve certain currently unsupported features
- Interesting thing about FITS is the numerous conventions developed by the user community for storing and transmitting various types of information. All these recognised conventions can be found here: https://fits.gsfc.nasa.gov/fits_registry.html
- The HIERARCH keyword convention (courtesy of Marten) which could allow having our own namespace to possibly incorporate all the unsupported Time metadata (format and vectorized time location to name a few) by using the YAML serialization implemented for ECSV
- The GreenBank convention which allows you to expand a standard keyword in FITS into a table column with a separate entry for each row. If you have a Binary Table with separate images as rows (probably multiple image exposures of an observation), each row could have a separate DATE-OBS and you COULD DO THAT. This amazing convention could possibly help me write a vectorized location in FITS (someone seems to be helping me 🙂 )
- Understanding the Chandra files’ time columns and the standards and conventions used there
- Looking at the io.fits code and particularly the astropy/io/fits/column.py module to get a head start regarding the high-level interface changes
Oh my god, I spoke too much 😦 I am currently busy ripping my hair off because FITS is harassing me. There are too many corner cases in the journal and I feel sad 😦 Also, the io.fits package is ugh (I am speechless wow!). But all in all, I am extremely happy with my progress (that is the only ray of light in my life right now, don’t take it away from me). My mentors appreciated my work during our last video call and they made my day (and the following days too 🙂 *they are too nice, sudden deep breaths*)
Oh and btw, I gathered all the courage that I possess and have started shifting to Python3 from 2.7 (yes I am one of those late-bloomers, okay that came out wrong). You should too, unless you want to pay to use 2.7 after 2020 (Python and paid NO WAY). I found the following extremely useful : https://snarky.ca/why-python-3-exists/
But wait I didn’t provide you a fun fact to ponder upon. So here is one. Most proper nouns used in Astronomy are acronyms. Moreover, in P. L. Lim’s words “We even use acronyms built from other acronyms.” Now you can die happily. Oh and “RR” in RR Lyr is actually just a numbering scheme. Now you definitely can die peacefully 🙂 *My work here is done*
So fellas, I have managed to ruin the past 20 minutes of your life. I will still end this by saying ENJOY and WAIT FOR MY NEXT POST! Adios!