One Month Since Leaving Canopy

Coffee Chats, Meetups, Tailwind, React, SAML, Eval, Extensive Prompt Engineering, and Replay Journal

It's been exactly one month since I left my full-time job at Canopy to focus on studying (mostly LLMs), building things for myself, and figuring out what's next. This first month flew by!

Ideation

I spent a lot of the first week on ideation, picking interesting things I can build alone while learning product engineering and practical LLM applications. I'm currently reading "Disciplined Entrepreneurship: 24 Steps to a Successful Startup," recommended by my friend Daniel Raffel. While not directly applicable to small solo/indie projects that I’m focused on now, it's a great resource for potential future things.

Introducing Replay

Most of my coding time this month was spent finishing Replay, a conversational journaling app I built for myself over the last few months. Journaling has been incredibly helpful for me, but I grew frustrated with existing apps. I believe LLMs can serve as a sounding board and provide context-aware probing. I'll write more about Replay in a separate post when I open it for signups. In the meantime, you can join the waitlist and see some features at replayjournal.ai.

Prompt Engineering

In Replay, I used OpenAI assistants for everything, from categorization and emotion analysis to mood detection, topic extraction, and summarization. I also have assistants for each of the 20+ journaling types and one for SEO. I spent a significant amount of time this month doing prompt engineering.

I did a lot of work on iterating on different types of summarizers for a future project.

Here are four helpful resources:

1. https://docs.anthropic.com/en/docs/prompt-engineering

2. https://platform.openai.com/docs/guides/prompt-engineering/strategy-give-models-time-to-think

3. https://www.promptingguide.ai

4. https://github.com/danielmiessler/fabric/tree/main/patterns (not a guide, but you can learn a lot from the source)

Rails Boilerplate + Tailwind

I spent a lot of time figuring out a good boilerplate for my next projects and learning Tailwind. The combination of Rails, Jumpstart Pro, and Tailwind (especially TailwindUI) has been a good choice. Tailwind has a weird learning curve and can be a pain for advanced things, but overall, I'm happy with it. Having everything in one file allows me to lean on a custom GPTs in ChatGPT and V0 for quick initial versions.

It's easy to underestimate the time spent on non-core tasks, but they always add up to weeks or look unprofessional if neglected (e.g., authentication, CDNs, servers, meta descriptions, landing pages, uploads, Stripe, etc.).

SAML + React

Last couple of weeks, I helped my friends at Koala with their SAML integration. I like their team, product and space, and I'm close with one of the founders. They're solving issues I had when helping sales at Canopy. And they have a React + Rails stack, that I wanted to see how they managed their frontend choices. They did it very pragmatically, handling the frontend quite well while benefiting from Rails on the backend (no GraphQL, model duplication or API calls). I'll write a separate post on their setup another day.

Canopy AI: Monitoring + Eval

This month, I also spent time upgrading and improving the AI leadership coach inside Canopy. It's been cool to monitor how people are using it, where it works well, and where it fails. I've started studying more about eval and assessing tools like Langsmith, Ragas, Guardrails, and OpenAI's framework. The problem is that all of these require a Python (or JS stack). So, next month, I'll likely roll out Athina.ai for Canopy to avoid introducing Python to the tech stack.

Meetups and Networking

One of the best things about being in San Francisco is the amount of people ot meet and events. I was negligent in staying connected with colleagues and friends while overworking at Canopy, but this month I started fixing that.

I also attended several meetups and virtual events, learning about AI agents, LLM-based crawling, accuracy, efficiency, RAGs, knowledge graphs, pre and post-processing, speed, and various architectures. My favorite meetups are hosted by the MLOps community, and I'm excited to attend their upcoming conference: https://www.aiqualityconference.com.

This month I also started asking for help from my girlfriend (she's an ML Engineer/Data Scientist). So she's sharing more notes from conferences, papers, etc. This paper in particular I find fascinating: https://arxiv.org/pdf/2308.10053.

Plans for Next Month: Eval + Agents

In the coming month, I plan to:

1. Open Replay for all users within the next week or two.

2. Study eval while measuring results using production data from Canopy and Replay, as well as during the prototyping phase.

3. Spend time studying both LLM theory and traditional ML.

4. Start working on a new agent-related product by the end of the month.

—

It's been an exciting and productive first month. Excited for the second!