I LOVE analyzing data. Deriving meaningful information data and visualizing it is something I absolutely enjoy doing.

So one evening, I decided it would be fun to export a group chat, analyze it, and send back the results on the group. It was a hit, yay! Everyone loved it and their feedback and suggestions helped me incorporate more features.

The analysis I did was VERY BASIC. And a lot of the code is umm, stolen borrowed from a dozens of tutorials and mixed it with some good ole elbow grease. Also it's worth noting that I CANNOT code; especially not in Python which is what I used. So it was a fun experiment. It's not something I technically coded. It's more like something I tried to code. It is extremely hacky .

1. Wrote some python code to take an exported WhatsApp group chat (whatsapp_chat.txt) to search and split the text based on the sender of the message. Used #regex for this. I know that Python regex and ECMAScript (which is the one I'm tiny bit familiar with) are different. The knowledge however, did not prevent me from making silly errors that took forever for me to recognize :D The whatsapp_organizesplit.py uses regex to extract messages from the main whatsapp_chat.txt and saves the messages into individual text files named after the person in a folder called “output”.

2. Used regex yet again to remove the timestamps. But before doing so, calculated the number of matches to find out the total number of messages. Saved this value. regex generator helped me to find an expression to match timestamps.

3. Now, we have text files for every participant. These have no timestamps. I calculated total words for each sender. I used the following python libraries to further process data.

   • wordcloud
   • textblob
   • nltk
   • emoji

4. Found out -

   • Most common words per person
   • Most common words for the entire group
   • Most common emojis per person
   • Most common emojis for the entire group
   • Sentiment / Polarity with simple sentiment analysis using textblob. I don't understand it at all but ticks all buzzwords which excited people on the group, so I'm happy :D

5. Saved all of these to text files.

Code is on Github.

Went through multiple iterations with the current version (1.0) being the first fully usable one imo. Finished it on May 24th, 2024. This is something I worked on during exams, in tiny breaks.

This version can do the following: – find matches (words/phrases), highlight the matches, and indicate total matches – find matches (regex), highlight matches, count them – can replace / substitute using regex – accepts not only a regular expression but also flags

(uses ECMAscript flavour of RegEx)

UPDATE: Code is on GitHub.

It's beautiful. I love it so much. It has a digital clock, a pedometer, a coin toss, and a drawing slate thingy. JUST LIKE THE GAMES.

Opened the .ino and hit compile on Arduino IDE. That's it!

I'm not sure why but the last time I tried it, it didn't work. I think I made some changes which fixed it. I replaced some libraries I think? Not sure if that's what helped because I was just messing around.

I just added

static LGFX lcd;
static LGFX_Sprite sprite(&lcd);

to the main.h. I don't even know if that's what made it work, or if I was just doing something wrong the first couple of times.

My money is on the latter. I'm too lazy to find out right now.

Why?

One of the reasons why I struggle to finish writing is because I'm constantly editing what I write. I know that editing is detrimental to writing. Get some stuff out there first. Get a block of marble first, and then you can chisel away.

From my site, This tiny web thingamajig is clean, no clutter, and lets you type without backspacing. Write first, edit later! Made a typo? That's okay, just keep moving forward. You can always fix it later. Write in a flow. Get the text out of your system and refine it later.

Uses

Can be used for journaling. I think this is a solid use case. Use it as a means of cathartic venting, void-shouting Can be used to write without editing. Like I said earlier, irrespective of the kind of writing, content is a prerequisite. You can edit your first paragraph a billion times and get a decent one, but now you're burnt out and there's pages to write.

Features

A word counter (that is slightly inaccurate but good enough) The ability to save what you've written as a text file Clean, clutter-free. Distraction free writing

Design

It is a minimal website that uses water.css, an open-source super-simple CSS framework! This makes the site look super clean and professional with zero effort from me.

Code

It's open source, of course! I should host it somewhere but until then, you can take a look / download the code from this Codepen link

UPDATE: It's now on GitHub.

I managed to find a scan of the first volume but as expected, it consists of images. The image format is jbig2 which is common for scanners. Tried to use OCR and extract text from the PDF using Tesseract. Tesseract doesn't support jbig2 format, so installed ocrmypdf. Thanks to Chocolatey, this was super simple.

After about 15ish minutes (my sense of time is wack so it could have been much more, or less), I had a PDF that I could highlight and index! I also got a text file with the extracted text. That's a win!