Gum

Well, the title of this blog post aptly describes my last month. For the Hindi speakers out there, you fellas understand the pun there, don’t you? For those who don’t, not to worry. Let me tell you a story. In a far far land made of candies, a young naive girl (probably the female version of Charlie in the chocolate factory or better yet the nasty girl there), was awarded the best chewing gum she could ever taste. It was the best thing that happened to her; she chewed it to her heart’s content. THE END.

Wait, where’s the catch in that story? Well, she was a stubborn young lady. The gum stuck to her (pun intended) and she couldn’t let go. It turned into those baseless, tasteless gums that you chew but can’t throw away because you can’t find a paper or a dustbin and you are too civilised to spit it out. (or you are one of those annoying kids who like to chew gum for no reason). But hey, there should be a happily ever after, eh? (Let’s say a thing like that exists). So, one day she worked on herself and made herself believe that it was time to let go and be happy. THE END.

Wait a minute, that just sounds inane and you must be wondering if I lost my mind. (sometimes I question that too). Well the chewing gum (my gum) has been my PR #6176.

I opened it on June 11th, being so proud of what I’d accomplished in such a short time. (well thank you). My mentors were happy seeing that I could get ~astropy.time.Time round-tripped so early. YES, I was that naive little girl frolicking around, content with one of the best things happening in my life. Never did I know that two months down the line, the PR would grow to a whopping 220 comments, 1,079 lines addition, about a century of code reviews, complete implementation and refactoring changes, 10 video calls with mentors and about hundreds of code and documentation builds. It has now turned to one of those tasteless chewing gums which you just want to get rid of AS SOON AS POSSIBLE. 

After my ardent efforts for the past two months, I have finally made the work in progress tag go away and finally this beauty is ready. My mentors and I have decided that once it gets merged, they’ll send me a bottle of champagne and we’ll celebrate (perks of having cool mentors). And I reckon today is the day. Eeeeeepppp. 

Nevertheless, I managed to complete another difficult task PR #6359. This involved expanding the FITS Column interface to allow coordinate type keywords (TC*) and their corresponding attributes in FITS Columns. This was necessary to incorporate column-by-column conversion of Table Columns into FITS columns (as suggested by @mhvk), which previously seemed like an impossible task, but turned out to be just strenuous. Valid FITS keywords have finally been added and validated. Meanwhile, I also ended up finding subtle bugs in ~astropy.io.fits and solving them. I finally, finally have started understanding the io.fits package and I must say this feeling is wonderful. Now my aversion/ineptness towards FITS has almost turned into sheer LOVE. Well, almost. (I finally beat you FITS, who says you are invincible). 

Although this chewing gum brought me a lot of gum (gum in hindi is sadness), I have learnt a great deal I must say. From writing beautiful code focused towards functional programming to refactoring it for memory and speed efficiency, understanding circular imports, writing customised, parameterised tests, decorators, documentation (sphinx is AWESOME! Although formatting is a pain), building docs (that took me a great deal of efforts for solving the platform oriented exceptions), switching to anaconda and python 3 (YEAH!) and understanding FITS (it can never be fully understood), these two months have been quite amazing. I think I might have read the four FITS WCS papers about a hundred times (not kidding). By the time I reach the end of the paper, I always feel confused and annoyed, but the intensity has decreased quite a lot (let’s just say it has).

While I work towards a career in astronomy, I am grateful that Astropy is now a part of my life. And although I am a little sad that GSoC will end soon (give me tissues please), I believe I will stay a part of Astropy, Always :). After all, the people are so warm and friendly. Who wouldn’t love to work with them? (Again not trying to flatter anyone 😉 ). Now I have a meeting to go to, so ADIOS fellas!

P.S. Sorry for the delay; I got caught up with a lot of multi-tasking this month :(. Also, you would have to adjust your eyes to the length of that blog post because THAT is totally unlike me.

 

Jargon, Jargon, sigh

My much awaited *DUE* blog post is here fellas (FINALLY). I must say that it gives me grave sadness that this post is going to be only about jargons and I am going to have to curb my humour to do so (which I believe I have, okay?). If I don’t, then the length of this blog post might climb the everest and inevitably a 128 bit floating point data type (when available and supported) won’t be able to count, in periods of caesium 133, the reading time of this post. If you didn’t understand this, READ MY PREVIOUS BLOG POST (publicising my own blog like a pro). OH BTW, if and when 128 bit floating point data type is supported by FITS, remember, I need to be informed FIRST. It is a matter of life and death I must tell you. *FITS, why you testing my patience levels? STAHP*.

Getting back to the point, the last two weeks I have been quite ambidextrous and I have successfully dabbled in multiple works (GSoC is included there I think. Just kidding, I NEED to pass my evaluation). While there is a marked difference in my vocabulary (thanks to GRE), the work for these two weeks has been more about understanding (a lot of it) accompanied by my beloved tea, rather than actual coding (irony eh?). I will talk about it in the following brief paragraphs (Don’t be so naive, “brief” is just a euphemism).

I never really officially introduced my GSoC project here. So here it goes *workaholic aarya in action*, a part of the project abstract:

Astropy has emerged as an integration of several independent packages (PyFits, asciitable), each responsible for an individual task in the storage, reduction and analysis pipeline of the astronomical data sets. It has mostly unified the interface in order to make these entities coalesce. However, the need for Astropy to shine as a single package and not just a collection of functionally independent modules, calls for seamless interoperability between the three special and powerful constituents of Astropy : coordinates, time and units and the underlying Astropy Table writers. We seek to develop a protocol that allows their storage, the​ ​mixin columns​, in FITS and ASCII ECSV, while still ensuring round-tripping.

Now the catch here is that, while I strive to achieve 100% round-tripping of the extensive metadata associated with the rich ~astropy.time.Time and ~astropy.coordinates.SkyCoord through FITS, abiding by the rules set by the FITS standard and having an implementation which is compliant with it and still manages to preserve complete information in that FITS header with limitations of 8 bytes keyword name, 80 bytes keyword record and a well-defined approach to every possible use-case that astronomy could face, is a great pain in the, well, everything that a human body possesses.

It might seem like FITS is actually inflexible, sorry for that portrayal, when in fact it is just the opposite. FITS (Flexible Image Transport System) is the data format most widely used within astronomy for transporting, analysing, and archiving scientific data files. FITS is much more than just another image format (such as JPG or GIF) and is primarily designed to store scientific data sets consisting of multidimensional arrays (images) and 2-dimensional tables organised into rows and columns of information. To put together everything that an astronomer ever longed in his entire life and to do so beautifully and with such flexibility for each person’s taste is truly incredible (I do not want to cry over this). To get a super-quick overview of FITS you could go to https://fits.gsfc.nasa.gov/fits_primer.html. There you will also find detailed documentation for almost about EVERYTHING (just exaggerating).

I talked about time in my last post, remember anything except my alluring humour? (well thank you). I did so because my current focus is to allow using the existing (but somewhat new) FITS standard for storing Time objects. I remember Tom saying the following to me during the proposal writing time, “This alone could take much of the summer but would be a very important contribution. It will be somewhat exacting work as the standard is long and detailed (22 journal pages), and does not precisely map to the astropy Time object (though mostly it is a reasonable match).” (I now truly understand why. *sobbing has slowly escalated to screaming*)

And if you really intrigued by my discussion regarding time scales in my previous blog, you should definitely read http://www.iausofa.org/sofa_ts_c.pdf. Hands down the best explanation you will ever find.

So I hope we are on the same page. I have to round-trip astropy Time and the Journal is looooooong (yes we are 🙂 )

The reason I am not getting into too many details is that it would take me hours to do so and I already speak too much (duh, I think we have already established that).

But to talk about jargon as I promised (damn it), here is the work that I accomplished until the start of last week:

  1. I managed to allow ~astropy.units.Quantity to be written to and read from FITS files, with relevant metadata of course, before the official GSoC period started. I also achieved the same for VOtable (I also did solve a couple of issues here and there and added a few minor features. FYI, I sometimes, with no reason, look at the contributors list and feel a little (a lot) exuberant seeing my name in the list *not creepy at all*)
  2. Time was really interesting to read about so I did so willingly. I read about time scales, got familiar with the norms and abbreviations, understood and played around with astropy Time and then finally read the “Representations of Time Coordinates in FITS” JOURNAL *sigh, I had to*
  3. For a detailed look at my work, you could visit https://github.com/astropy/astropy/pull/6176 and try and understand what is going on there (If you have the patience, anything is truly possible. Except writing your own fits module 😉 For those who did, what did you eat? Patience for dinner?) I have managed to round-trip Time through FITS to quite an extent. I still need to incorporate certain features not directly supported by the FITS standard, for eg. vectorized location

Then the last week and a half were jam packed doing the following:

  1. Cleaning the code according to all the reviews
  2. Bringing together everything related to the FITS time coordinate frame in one place, the FITS_time class, in which case the knowledge of the names of the keywords could be kept in one place (My local tests run, eeeeppp)
  3. Rigorously testing and checking for unusual corner cases (I have created a separate FITS_time test class to do so)
  4. Reading the FITS journal again to skim through a few topics again to understand the possible use-cases and to find keywords which could possibly solve certain currently unsupported features
  5. Interesting thing about FITS is the numerous conventions developed by the user community for storing and transmitting various types of information. All these recognised conventions can be found here: https://fits.gsfc.nasa.gov/fits_registry.html
  6. The HIERARCH keyword convention (courtesy of Marten) which could allow having our own namespace to possibly incorporate all the unsupported Time metadata (format and vectorized time location to name a few) by using the YAML serialization implemented for ECSV
  7. The GreenBank convention which allows you to expand a standard keyword in FITS into a table column with a separate entry for each row. If you have a Binary Table with separate images as rows (probably multiple image exposures of an observation), each row could have a separate DATE-OBS and you COULD DO THAT. This amazing convention could possibly help me write a vectorized location in FITS (someone seems to be helping me 🙂 )
  8. Understanding the Chandra files’ time columns and the standards and conventions used there
  9. Looking at the io.fits code and particularly the astropy/io/fits/column.py module to get a head start regarding the high-level interface changes

Oh my god, I spoke too much 😦 I am currently busy ripping my hair off because FITS is harassing me. There are too many corner cases in the journal and I feel sad 😦 Also, the io.fits package is ugh (I am speechless wow!). But all in all, I am extremely happy with my progress (that is the only ray of light in my life right now, don’t take it away from me). My mentors appreciated my work during our last video call and they made my day (and the following days too 🙂 *they are too nice, sudden deep breaths*)

Oh and btw, I gathered all the courage that I possess and have started shifting to Python3 from 2.7 (yes I am one of those late-bloomers, okay that came out wrong). You should too, unless you want to pay to use 2.7 after 2020 (Python and paid NO WAY). I found the following extremely useful : https://snarky.ca/why-python-3-exists/

But wait I didn’t provide you a fun fact to ponder upon. So here is one. Most proper nouns used in Astronomy are acronyms. Moreover, in P. L. Lim’s words “We even use acronyms built from other acronyms.” Now you can die happily. Oh and “RR” in RR Lyr is actually just a numbering scheme. Now you definitely can die peacefully 🙂 *My work here is done*

So fellas, I have managed to ruin the past 20 minutes of your life. I will still end this by saying ENJOY and WAIT FOR MY NEXT POST! Adios!

 

Getting Through GSoC (I guess so)

There are two types of people in this world. The ones who are well with their words and the ones who simply aren’t. Yes, I am referring to the sociable “Irene Adler”s and the mysterious “Sherlock Holmes”s out there. Wait a second, that doesn’t sound quite right. Have you heard that there is always an exception to the rule? Well in this case, there is me, falling in neither category. A testified grandiloquent who can blabber on so much, that by the time you finish reading this, you would probably lose faith in humanity.

Hence,

DISCLAIMER: Those with low patience levels / ADHD / who are prone to being choleric, do not say that I didn’t warn you about the length of this text. Although I must say that you will definitely enjoy reading it (that is what I will be telling myself).  I have been told that I get distracted way too easily. So let me get back on track. I am rather late in posting all of this, but I can assure you that I have been extremely punctual with my work (it is not an option when you take on such herculean tasks). Moving on, I will be rewinding a little bit, to sort of like have a flashback of the past month, solely for the sake of my satisfaction.

The GSoC acceptance day went by with the following exclamatory statements, “OH MY GOD!” *RING* *RING* “I GOT IN!” *RING* *RING* “DID I?” *RING* *RING* “I GUESS SO!” *RING* *RING* “Me to me: STOP IT AARYA, WILL YOU?” and a little bit of dancing around (probably a lot of it). The following days were quite ecstatic, characterised by a lot of galloping around here and there.

Then, the much awaited semester exams (awaited to end) came head on and I should say I got through it (let us all hope that I did). My mentors were amazingly understanding during these days of my woe and I am so happy to have them (not trying to flatter them, really). After a day or two of relief, I eventually had to get back to reality. However, I am the kind of person who hates “not” being busy, so for me it was a win win situation.

Then came the real obstacle, the “Representations of Time Coordinates in FITS” Journal with 22 pages of flexibility for the astronomical community and months of headache for me 🙂 I had to learn about Time, the way it is contemplated by astronomers round the world and the plethora of options available in order to do so.

Calculations in any scientific discipline may involve precise time, but what sets astronomy apart is the number and variety of time scales that have to be used. Although the era of new and precise “Atomic Time” scales is emerging, astronomy continues to deal with the very phenomena that lie behind the conventionally used but quite obsolete time scales, in particular the rotation of the Earth and the motion of the planets. Thus, we have our own fossil record of time scales, carbon-dating to the evolution of humans (okay that went a little too far, but I guess you get the point).

Now, you might be wondering what a time scale is. It is simply a specification for measuring time: either the rate at which it passes or points in time; or both. But WHY DO WE NEED IT? Well, the very misconception lies in the faith that a day consists of exactly 86,400 seconds. If we consider the atomic clocks round the world, which are also the official time-keeping standards, they do run at this rate. But the UTC time that we all follow so blindly, does not exactly do so. To my surprise, there is a concept of “leap seconds” where a second is added to a day once in a while to sort of like stop the clock for a while so that it catches up with the earth’s rotation. Why don’t they add multiple such seconds before the day of my exam? (eternal woes of life)

If we could all recollect a little bit of physics that we have learnt so far, we would realise that it all depends on the context or the reference frame. Whether time is based on the Earth’s rotation, which is irregular due to various reasons, making it unsuitable for several cases, or it is the “proper” time at our exact location, it all really does matter. After all, precision might be the better half of an astronomer’s life (probably because the universe makes it so difficult to have this idealistic thought). The concept of absolute time, as we all know, has been scraped off by General Relativity and it does make a difference whether your calculations are associated with the Geocentre (The Earth) or the Barycentre (The Sun) or just an ordinary observatory location. Frankly, I feel a sense of enlightenment when I talk about the universe and things like general relativity (partly because I find it extremely cool). But we don’t want me to die while writing this. I will thus conclude this part by saying that time is, in itself, an extremely intriguing concept. A big shout out to GSoC for making me understand it all so vividly.

The weekly conference calls with my mentors are when I actually unfold the extent of this problem and they have been rather amazing I must tell you. I have fairly restrained myself from being this chatty there, as I do not want them to miss their meetings (I am very much capable of doing so) but I think that will soon change.

Summing up, the community bonding period and the first week of official coding passed by rather too quickly.  To be honest, the golden embellishment to all this “Time Jargon” has been this one statement, that I made sure I tell each and every soul I possibly know or could know, and knowing it has made me so very proud of myself. Don’t worry, I will let you know of this knowledge as well. Here it goes,

“The duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium 133 atom”, the latter being at 0 K and in zero magnetic field stands for, yes the SI SECOND. *SIGH* This is where I will leave you thinking and wondering about how your entire life has been a lie (please don’t take that too seriously).

Eventually, I did end up condensing the 22 Pages FITS journal into a 4 page vital summary (which took so much time and I believe I did a fantastic job, do not dare say otherwise, *takes a bow*). I also understood the various obstacles that are involved in this project and how each of them contributes to the difficulty of the problem. Quoting my mentors, “This has been an issue for years and people say it is easy. Well, we know it is not”. (they truly understand me, *sobs a little*).

Coming to the technical part, the project is mainly focused on the long-standing issue #3000. This is one part of the aimed seamless interoperability between the three powerful components of AstroPy : coordinates, time and units and the underlying AstroPy Table writers, particularly FITS. So, my task is to allow Time columns in Astropy Tables to be written to FITS as Binary Tables and reading them back with an idealistic goal of 100% round-tripping (that, my friend, is very difficult but I will get there, one day). This would also include reading existing FITS files supporting time coordinate columns which is a pretty tricky job.

I have currently opened a Pull Request which took an arduous effort to make it what it is today. Well, that was bound to happen, since the io.fits sub-package code has made me want to pull my hair out (not kidding). It is a beautifully written code, no doubt, and the flow of control is impeccably organised into multiple cleanly designed classes. But, that just makes my job harder, doesn’t it?

The PR introduces a time module within io.fits, as a preliminary version to write and read AstroPy Tables containing Time columns to and from FITS files. Currently, I have taken care of writing Time by writing it as a (N,2) column with each element as a vector (jd1, jd2) and an explicit AstroPy format keyword. Considerable round-tripping of AstroPy written “FITS Binary Tables containing time coordinate columns” has been achieved. This includes metadata : scale, format and singular location. I have also started work on reading generic FITS Binary Table files, by generic I mean not written by AstroPy.

I reckon it is always the small things in life that make a difference. I cannot express how happy I feel looking at the code that I have produced within the past week or so. It might be simple (I would definitely falsify this if you try to say so), but it is something that I have really loved doing and it does make me utterly butterly happy (in spite of all the cribbing that you just had to go through).

Just because I am too attached to this link for now, I am posting it here for no reason whatsoever : https://github.com/astropy/astropy/pull/6176

Now I have a few failures that I need to solve, because travis never lets you be calm. 🙂 It is also 4 am here and I guess I need to get some sleep. You should be happy, you can finally go now. Or did my idiosyncrasies appeal to you to the point that you would want to listen to more of this? (I will hope that is true). Thus, this is how fast time passes by (maybe the scales aren’t so accurate after all) and that is how people get through. I got through two amazing weeks of coding and I am looking forward to the rest. Till then, Adios mate and wait for my next blog post!

 

 

 

Google Summer of Code

I am the kind of person who loves writing. So I would first like to acknowledge the fact that GSOC gave me another medium to do so.

Now the purpose of this blog is Google Summer of Code. To talk about my project, my experience with GSOC and fingers crossed, if I do get selected, talking about my work throughout the summer. I believe words have a lot of power in them, be it in conversations or in codes. Just as I look forward to working in the world of open source, I would like my thoughts to be so too.

So for me, the procedure for GSOC started from checking ideas posted under the OpenAstronomy umbrella, specially those under Astropy. It was rather an obvious choice for someone who loves astronomy and is somewhat a computer geek. I stumbled upon the project “Seamless Combination of SkyCoord, WCS, Table and FITS”. I read a little about its description and I could actually relate to the issue that the project wanted to tackle. And somehow it made this amateur astronomer a little happy. Not just because it made me feel like I have stepped into the field of astronomy (a little bit yes), but also because it intrigued me to the point that I happened to read about it for hours together. FITS and I go a long way, and I am happy to say it hasn’t disappointed me (yet).

An excerpt from my story

It is rare that you find such joy in something, that you cannot explain the very existence of it. I love gazing at the stars, pondering upon the basic questions of nature, I love understanding the universe, believing in the energy of it and I yet hate to not have the answers to all of it. It has been rather paradoxical, the stream of my thoughts, but it is why I want to spend my life trying to comprehend it.

I was introduced to night sky observations at a very young age, thanks to my father’s love for the stars, which I can now proudly say runs in our genes. Being brought up in a city, I used to eagerly wait for the summers to travel to my native place, a small village nearby, to be away from the urban lights. My father and I were once sitting on the roof of our quaint little country cottage, gazing at the heavens above, in awe of a canvas of black and white. I simply asked him, “Could there be a unified theory to explain it all?”. He said he didn’t know. So I penned it down, my first ever literary work, probably thinking someday someone might answer it. I found a way to my lingual glories and later discovered that I wanted to be that someone, finding answers to the fundamentals of nature.

It has been rather stereotypical to either be a hopeless romantic of the arts or to develop a scientific skepticism, but I’ve never been a stickler for stereotypes. I like to believe great science requires greater imagination. In school, I developed an affinity towards physics, because it fed my curiosity of the clockwork of the universe and mathematics; the language of physics. Starting with the fundamentals by Halliday & Resnick, I slowly levitated towards Hawking, Feynman and Griffith. These weren’t just ordinary textbooks, for me they were intellectually enticing works of art on how the world works in the physics domain. I was always more of a self learner and somewhat of a teacher. I loved discovering new things and sharing my enthusiasm with like minded people. Nothing gave me greater thrill than solving an intriguing problem. It paid its due when I was awarded a scholarship by the Government of India for securing a rank in their examination. This began my journey in the pursuit of science as a career. During this time, I also involved myself in sketching and writing as a hobby. In addition to representing my state in the national level drawing competition, I started writing poems for my school plays. Recently, I have developed a new passion in blogging. This has been a perfect coalescence of my individuality.

When in junior college, one of my friends asked me to be his partner in a programming competition. I did not know how to code, but the problems were enticing and those three hours transformed my life. I was absolutely amazed at how a difficult problem could be solved by tackling one logical component at a time, at least if it’s not an NP complete problem. This resembled the way the modules of the universe are governed by the fundamental laws of physics, and I knew just what I wanted to do.

I am currently a third-year student at Pune Institute of Computer Technology (India) majoring in computer engineering and I want to work in the field of computational astrophysics/physics. It took me some time to realise the interdisciplinary nature of the fields of science, but I’m glad I did.