GLM 5.3 Ate my Light Sail

The Brief

As I mentioned in the last post I decided to replace AWS LightSail Wordpress server with a static site. Not because LightSail was expensive but more because Wordpress was turning into a bit of a pain, it needed upgrading and I just don’t need all the features it offers. It also cost $9 pcm, and the only benefit I could see anyone getting from that was spammers. But, one upside is that I’ve now got 12 years of SEO spam in my MySQL backups that I can analyse with AI! A project I’ve always wanted to do but never had time, so stay tuned for that.

Anyway, when you think about building a static site there’s actually quite a lot of tech stack to grok before you can actually get something live. Sure I can probably do all those things from scratch but I do them rarely enough that it would be a lot of looking stuff up on the internet to remind myself. And that’s not why I’m here, I’m here to write words - not code.

So this is the shopping list for what I think a static site needs to have:

1. A repository of posts in Markdown that are stored in a cloud based code repository.
2. A code site generator, that can take repository check-ins and publish them (à la CI/CD).
3. A bare-bones visit tracking tool that gives me a feel for interest in what I’m writing (I’m only human after all) without the horror show of Google Analytics.
4. An online storage for the tracking data that is hopefully free at low volumes.
5. Style: a banner logo (fresh design), a favicon, a font and colour scheme - i.e. wizard hat, green screen …
6. An HTTPS hosting solution (preferably free) capable of handling the requirements of above
7. A suite of built in RSS feeds
8. Infrastructure as code so I can apply all the changes from the command line, without having to fight my way through AWS’s web consoles.

There’s a lot of skills you need to have learned in order to do the above quickly from web developer, to web designer and infrastructure engineer.

Here’s what I want to disprove:

GLM-5.3 is not capable or cheap enough for a real world design, development and engineering task. Where ‘capable enough’ is compared to frontier models; and ‘cheap enough’ is less than one month of the cost of LightSail.

Why choose GLM-5.3? We’re already seeing that Anthropic’s revenue growth whilst still impressive has disappointed recently. The winner will likely be an actor with less debt and lower costs that can deliver AI cheaply at scale. That winner will emerge when the capability is good enough and cheap enough to gain an advantage.

To be clear, I have no affiliation with Z.ai, it just seemed like a capable and cheap model to try.

Spoiler GLM-5.3 was mostly good enough. Some things went a bit sideways, the pain was self inflicted and the costs were minimal. What surprised me was that GLM-5.3’s suggestions regarding my AWS issues were better than the paid AWS support. Which is mind-blowing.

The Pitch

So armed with pretty much this spec I fired up my Claude Code (attached to OpenRouter, no affiliation) and chose the GLM-5.3 model.

Here is what went wrong:

Artless

I hit the first issue trying to create the Wizard hat logo, the first attempt was pretty bad, but I knew what I wanted so I assumed I could iterate with GLM-5.3, turns out that you can’t really. There is a model for that but I didn’t actually go hunting for it and I cheated and switched to Sonnet-5-5. Which is still not frontier, but is capable of doing what I needed. That experience was still a bit painful but eventually I got something I was happy with, in hindsight I should’ve switched to GLM-Image, but it doesn’t really matter. It had shown me an important limitation.

Soulless

My generated Terraform wouldn’t apply because I kept getting a certificate error. Now this is where things got really interesting … the reason it wouldn’t apply is that I hadn’t properly taken down my AWS LightSail instance whilst I was building this and deep inside LightSail it had created a CloudFront Distribution that bizarrely doesn’t show in the AWS console or API calls. I spent a lot of time looking at it, fiddling with DNS entries in Route 53 to no avail.

I couldn’t figure it out, all the internet articles pointed me to the same three steps to find the CloudFront distribution that AWS said didn’t exist. That is obviously AI generated so I felt like I needed more.

So I opened an AWS support ticket in the free-tier that pointed me to the same three articles. Broken and sad I opened a $30 monthly subscription to AWS paid support, and opened a new ticket. It generated the same results except for the AI was slightly better, it incorporated some mention of what I’d written in the answer. But essentially the same output. I thought about this, and tried a few more times with GLM-5.3 to get it working and that’s when GLM mentioned that LightSail had a hidden CloudFront distribution embedded in it! It took about 5 mins to find, delete and then the site was deployed.

The Win

It’s hard to know how this experience could’ve been much better if I’m honest. I couldn’t have expected GLM to know about my LightSail issues because LightSail is itself a pretty big product and I didn’t give my harness very many IAM permissions to do its job.

One thing I would note is that I did have to add some technical expertise at times, some examples:

• The default AWS CloudFront selections were not what were in my requirements. I had to iterate a few times (with GLM-5.3) to adjust the terraform to remove some choices it had made. One of those iterations removed the TTL (which can’t be set in the free tier) …
• After doing the above the default (time-to-live) caching of CloudFront means that new publications take an entire day to reach the edge, I decided to add something to the build pipeline to invalidate updates and force new pages out sooner, so that tracking aligns with publication.
• Cross-linking between a markdown repo and the website needs special handling I found that through manual site testing and feedback.

Perhaps the biggest surprise in all this was the quality (or lack-of) from the SVG generation of a battered green wizard's hat, there’s no pelican on a bicycle here this is something that should be relatively easy (compared to a pelican on a bicycle) to draw. But as I said that (in the end) was mostly Sonnet-5-5 so doesn’t massively change my hypothesis it’s still not frontier.

The Science-y bit

Here’s where the money went:

So I didn’t fully reject my null hypothesis. It was both more expensive (by $1.12) and the design part was still awful. But I have to say it was pretty damn close.