Lost some files in AWS Glacier? I found ’em
During my last few visits to the AWS Popup loft, I really dug into my Amazon Web Services account, especially the billing. My bill had skyrocketed to over $5 a month! What had happened? How did I miss this huge spike in billing? Well, it turns out that a majority of that bill was from AWS Glacier, I must have stuck something in there for safekeeping and forgotten about it.
The biggest “issue” I have with the AWS S3 service (and therefore glacier) is that you have no search in the web console. Which means I’d have to flip through every single bucket, every single folder, and check the “storage class” on every single object by hand. That sounds obnoxious (I have lots of stuff in S3) and a really good task to be outsourced to a program. Since Node.JS is my language of choice, I wrote a quick script to find every file in S3 that was in my Glacier.
Allow me to present the aptly named “find-glacier“.
It’s a very simple app, the instructions on how to get started are in the README.md file. It will either list EVERY object you have in S3 (in case that’s useful to you), or list just the Glacier objects. In my case I found and deleted the decayed objects that were taking up space/money, problem solved.
The script does work with S3 buckets in any region, which posed an interesting challenge. Due to the way AWS is setup, it has to connect to every region you have buckets in (handled automatically). The other interesting piece was learning how to use the ‘q’ library which is “A library for promises”. It helped me massively by staying out of the “pyramid of doom” (as they call it) and taught me about the proper use of asynchronous calls, tracking embedded async calls, etc. It’s not a terribly long script, but it was an interesting project to cut my teeth on. Hopefully, it helps someone else with the same “where is that ##$$ file” problem I had.
