Month 2
Lots of papers, coffee chats, contract work, and conferences…
I've been away for a couple of weeks, spent probably 30% of my time networking and meeting people, and the rest helping my friends at Koala. This meant missing two weekly posts and delaying the public release of Replay. The app is working well, but I’m having a tough time finding time for the last details and GTM.
While I haven't done much hands-on AI engineering this last two weeks, I've been keeping up with theory. Here's some of the notes:
Daily Prompting Paper
For a while now, I've been checking Arxiv daily for new papers on Prompt Engineering. It's a topic that's usually easy to understand, even in academic papers, so great way to kill time between tasks.
Many poor results in AI come from inadequate prompting skills + techniques like prompt chaining and retries, weren't practical before, but now make sense.
In June, this excellent paper was published. I spent the last couple of weeks reading it thoroughly, along with many of its citations. I've compiled some notes here, and I'm also planning to create a better structured resource with more examples and additional notes I didn't include.
Meetups and Conferences
I've been attending several meetups and conferences, including AIQCon yesterday. It's interesting to see the growing focus on AI Engineering (meaning implementing GenAI in products). Last year, there a lot of interest in LLMs themselves, but now the emphasis is on making LLMs perform well in real-world applications.
Key topics include:
RAG (Retrieval-Augmented Generation): Techniques to improving embeddings, content augmentation, and advanced techniques like GraphRAG.
Evaluation: Setting up good test suites, deciding what to test, and determining who should write the tests.
No fine-tuning: The consensus is to focus on prompting, RAG, and evaluation before considering fine-tuning. Lot of pain and scar from fine-tuning already.
While some events feel like they have a high noise-to-signal ratio, I always learn a lot as a newcomer. Also, it’s such a great time to live in SF. For future conferences, I plan to research the participants and their topics more thoroughly beforehand.
A useful resource from yesterday's conference: https://huyenchip.com/llama-police.html by Chip Huyen (well worth following on X).
Books
On the topic of learning more theory, I'm currently reading two books:
1. Programming Machine Learning
https://www.amazon.com/Programming-Machine-Learning-Zero-Deep/dp/1680506609
I first read this when it was released, but I've forgotten most of it. Although it's a bit outdated and pre-LLM, I like the author's writting. Given how close AI Engineering is to traditional ML, I want to revisit this material.
2. Neural Nets mini-course by 3Blue1Brown
Another basic entry level material I was revisiting this week is this mini-course on neural nets: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi
2. Oreilly’s AI Engineering book
https://www.oreilly.com/library/view/ai-engineering/9781098166298/cover.html
This book was recommended by a close person, and I attended a talk by the author yesterday - it was one of my favorites, despite being too short. Only three chapters are available now, so I'll keep an eye out for future updates.
