I Got Tired of Debugging Curl at 2 AM, So I Built a CLI
Tired of fragile 2 AM curl commands? Learn how a custom CLI for your API reduces errors, speeds up incident response, and makes on-call debugging safer.
Join the DZone community and get the full member experience.
Join For FreeIf your team owns online API endpoints, chances are you — or someone on your on-call rotation — runs curl commands a lot. Curl is a fantastic tool: it's tiny, ubiquitous, and scriptable. But when you're bleary-eyed at 2 AM, it can be too easy to make mistakes with curl. Which header did I forget this time? Did I remember to URL-encode that JSON field? What was the exact syntax for the authorization token? And how do I reliably pipe the result from one command into another without mangling it?
Picture this scenario: It's the middle of the night, and an incident has kicked you out of bed. You’re troubleshooting an API issue with curl commands. First, you need to fetch a user session via a GET request, then use that session ID in a follow-up POST request to revoke the session. In your half-asleep state, you might do something like this:
# First, fetch session data for user 42 (simplified example):
curl -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" \
"https://api.mycompany.com/sessions?user=42"
# ... parse the JSON output to find the session ID, then:
# Second, revoke that session by ID:
curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" \
-d '{"confirm": true}' "https://api.mycompany.com/sessions/SESSION_ID_TO_REVOKE"
Even in this simple two-step workflow, there are plenty of opportunities for error: forgetting a header, miscopying the session ID, or not encoding parameters correctly. Under pressure and lack of sleep, these mistakes happen more often than we'd like. That’s when I decided there had to be a better way.
Instead of memorizing perfect curl flags and manually chaining commands in every incident, I invested a little time in building a small, well-designed custom CLI tool for our service. Now, when I need to perform those tasks, I run a friendly command like:
mycli get-session --user 42 --json | mycli revoke-session --from-stdin
This custom CLI encapsulates the nitty-gritty details and does the right thing automatically: setting the proper headers, encoding values, handling retries, logging actions, and safely managing secrets. The difference at 2 AM is night and day — fewer errors, faster investigation, and far less frustration. In this article, I'll explain why a custom CLI for your API is worth the effort, and share some practical design tips (with examples) to build one. You don't need to be a senior wizard or spend weeks on it; even a tiny Python script can evolve into a lifesaver on your next on-call night.
Why a Custom CLI Is Worth It
Building a custom CLI might sound like extra work, but it pays off quickly. Here are a few big benefits we discovered:
- Reduce human error: The CLI can enforce all the required bits of a request, so you don't forget them under pressure. It automatically adds headers like Content-Type: application/json, URL-encodes query parameters, and fills in sensible defaults. This means you're much less likely to omit a critical option or type something incorrectly when you're in a hurry or groggy. By letting the tool handle the boilerplate, you avoid those “oops, I missed a header” moments.
- Speed during incidents: On-call investigations often require making multiple calls and then chaining their results. For example, you might get a resource, extract an ID from its output, then use that ID in another request. Doing this by hand with curl means lots of copy-paste and chances to slip up. A well-designed CLI can streamline this by outputting machine-friendly data (like JSON) and even providing helpers to pipe information from one command to the next reliably. This dramatically speeds up incident response when every minute counts.
- Standardization across the team: With a custom CLI, everyone on the team uses the same commands to accomplish tasks. No more “Did you use curl with
-vor with that special script?” or having five slightly different shell scripts floating around. The CLI becomes the single, consistent interface for your service. This consistency means fewer variations in runbooks and support tickets — if one person solved an issue with mycli get-user, anyone can follow those exact steps next time. - Auditability and safety: Your CLI can have logging and safety features baked in. Every command run could log an entry (to a file or monitoring system) with what was done and when — invaluable for post-incident reviews. You can include a
--dry-runmode to show what would happen without actually making changes, and require explicit confirmation (like--yesflags or interactive prompts) for destructive operations. These features help prevent accidents (no one wants to delete the wrong data at 2 AM) and provide an audit trail of what calls were made during an incident.
In short, a custom CLI turns those ad-hoc curl calls into something repeatable, dependable, and user-friendly. It reduces the cognitive load on engineers during stressful situations. Next, let's look at how to design such a CLI effectively, based on what worked for us.
Practical Design and Best Practices
When building your CLI tool, you want it to remain simple yet powerful. Through trial and error, I arrived at several best practices that made our CLI a success. Here are some of the key design principles, along with examples from our journey:
1. Keep It Small and Composable
Design your CLI around focused, short commands that can work together. Each command should do one thing well – like get-session, revoke-session, get-user, delete-user, etc. By making the commands composable, users can pipe JSON output from one into another, leveraging the Unix philosophy. For example, our tool lets me fetch a session and revoke it in one line by piping:
mycli get-session --user 42 --json | mycli revoke-session --from-stdin
In this example, get-session returns session data in JSON, and revoke-session can read the necessary information from standard input (the output of the previous command). This beats writing temporary files or copying IDs manually. Keep each sub-command simple, but allow advanced workflows by chaining them.
2. Make Defaults Explicit but Overridable
Your CLI should provide smart defaults for things like timeouts, headers, or API host URLs so that, in most cases, users don't need to specify them. However, when a non-standard scenario comes up, let those defaults be overridden via command-line flags or environment variables. For instance, the default request timeout might be 5 seconds — fine 99% of the time — but if an engineer knows an endpoint can take longer, they might use --timeout 15 for that call. Similarly, you might default to hitting the production API, but allow an --env staging flag or an environment variable to easily switch to a staging server. Explicit defaults make the CLI predictable, and overrides make it flexible when needed.
3. Handle Encoding and Quoting for the User
One huge advantage of a CLI wrapper is that it can automatically take care of those pesky encoding issues. URL-encode path and query parameters behind the scenes, so that spaces, special characters, or Unicode in input values won't break the request. Also, handle quoting and escaping if your CLI prints or logs an equivalent curl command.
For example, if your CLI has a debug mode that outputs “Here’s the curl command we’re running,” ensure that any quotes in JSON bodies are escaped properly, API keys aren’t exposed, and the command is copy-paste ready. This way, users can trust that whatever data they pass, the CLI will transmit it correctly to the server (no more manual percent-encoding or quote juggling).
4. Safely Manage Secrets
On-call scripts often involve tokens, API keys, or passwords. Build your CLI to source credentials from secure locations — whether it's the OS keyring (the operating system’s secure credential store), a secret management service like HashiCorp Vault, or environment variables set by the user. The key is to avoid ever printing secrets to the screen or storing them in logs. Be mindful even in verbose or debug modes: implement a mechanism to redact or mask sensitive fields (e.g., show API_KEY=**** in logs). This ensures that if someone scrolls back through their terminal or shares a log, they won't accidentally expose confidential info. It's a small thing that builds trust in using the CLI for real production systems.
5. Offer Friendly Output Modes
Different situations call for different output formats. Some days you’ll run a command in a terminal and just want to read a nicely formatted result; other times you’ll want to pipe the output into another program for further processing. Support multiple output modes to cater to these needs. For example, we added:
- A
--jsonflag for raw JSON output, making it easy to pipe into tools like jq or another mycli command. This mode strips out extra fluff and just prints machine-readable JSON. - A
--pretty(or human-readable) mode that formats the output with indentation, colors, or ASCII tables – great for quick eyeballing of data during a live debug. - A
--curlflag that prints the equivalent curl command the CLI would execute, instead of actually making the request. This is super useful for debugging or learning — it shows exactly what the CLI is doing under the hood. New team members loved using--curlto understand the API calls, and it was also handy when we needed to share a one-off snippet with someone who didn't have the CLI installed.
By providing these modes, the CLI caters to both humans and machines, making it a more versatile tool.
6. Include Dry-Run and Confirmation for Destructive Ops
When a command is going to make a destructive change (like deleting data, shutting down a service, etc.), it's wise to build in safeguards. One pattern we used was a --dry-run option that, when enabled, would skip the actual API call and instead print out what it would do. This lets an engineer double-check, for example, “Am I about to delete the right user account?” before pulling the trigger.
Additionally, for truly dangerous actions, consider requiring a confirmation step – for instance, the command might prompt “Are you sure? Type YES to proceed,” or require a --yes flag to be provided explicitly. This extra step prevents accidents, especially when you're in a hurry. It's much better to add one more step than to explain to your boss why a production resource got zapped unintentionally.
7. Handle Errors, Retries, and Logging Gracefully
Real-world networks are flaky; APIs can timeout or throw transient errors. Your CLI can make life easier by handling these in a consistent way. Implement a retry mechanism for transient failures (with sensible backoff, maybe configurable retry counts). This way, if an API call fails due to a temporary glitch, the CLI will automatically try a couple of times again — often saving you from waking someone else up or manually rerunning the command.
Also, make sure to exit with meaningful exit codes (and messages) so that if the CLI is used in scripts, the script can tell success vs failure. For observability, consider logging each request and response status (without sensitive data) to a debug log file. That log can be gold for troubleshooting later: you can see what happened during that 2 AM incident, what failed, and how long each request took. Structured logs (e.g., in JSON lines format) are even better for feeding into analysis tools or dashboards during post-incident reviews.
8. Ship It as a Real Tool (Packaging and Distribution)
Treat your CLI as a real piece of software, not just a one-off script. If you're using Python, you can package it as a Python package (for example, create a wheel or an installable module) and define console entry points so it can be installed and run with a simple command (e.g., pip install mycli gives you the mycli command). Other languages have their equivalents (publishing a Go binary, an npm package for Node, etc.).
The goal is to make installation and updates easy for your team. At my company, we set up an internal PyPI server to host our CLI package so any engineer could install or update it via pip. Automate the release process as much as possible: for instance, have your CI/CD pipeline build the package and push it to the repository whenever you tag a new release. This ensures the CLI is consistently available and up-to-date across all the machines that might need it.
9. Don't Skimp on Testing and Docs
Even though this CLI is primarily an internal tool for debugging, it’s still software that benefits from testing. Write unit tests for the critical pieces – for example, test that the CLI correctly builds a request given certain inputs (proper URL encoding, headers set, etc.), and that it properly parses responses or errors. If possible, set up a few integration tests against a staging or mock server to exercise the real HTTP calls end-to-end (you don’t want the first time you discover a bug to be during a production incident!).
Documentation is equally important: keep a one-page README or cheat sheet with common usage examples. We put our CLI’s README in our team wiki and included examples like “How to quickly fetch and revoke a user session.” This doc was a lifesaver for new on-call engineers at 3 AM – they could copy-paste the example commands and adapt as needed, with confidence that the CLI would handle the rest.
Following these best practices, we ended up with a CLI that our whole team came to rely on. It started small — just a couple of commands — but over time we added more as new needs arose. The key was that we built it around actual pain points we felt, which brings me to the final tip.
Deploying and Distributing Your CLI
Designing a great CLI is half the battle – you also need to get it in the hands of your team (and keep it working on the systems where it's needed). Here are some practical tips for deploying your custom CLI tool across your environment:
Continuous Builds and Internal Distribution
Set up your CI pipeline to build the CLI on each release (for example, generate a Python wheel or binary executable) and publish it to an internal package repository. In our case, we used an internal PyPI server for the Python CLI. This way, installation is as easy as pip install mycli for any engineer, and updates are just as easy. Having a central distribution point beats everyone copying scripts around or cloning git repos manually.
Automated Installation on Key Systems
If your on-call workflow involves jumping onto special bastion hosts or debug servers, ensure the CLI is readily available there. We leveraged configuration management tools (like Ansible and Salt) to automatically install or update the CLI on all our dev, staging, and production bastion boxes. You could also provide a Docker container or bundle it into a Kubernetes debugging pod if that’s how your team rolls. The point is to eliminate friction — when an incident strikes, no one should be scrambling to set up the debugging tool; it should already be on the system waiting to be used.
Versioning and Change Management
Treat the CLI like a product. Use semantic versioning (SemVer) for releases (e.g., 1.0.1, 1.1.0, 2.0.0) and be strict about it. If you introduce a breaking change to how a command works or deprecate something, bump the major version and communicate it clearly to the team. We would announce updates in our team channel and update the incident runbook with the new version info. Tagging releases in your source control and writing brief release notes (even just in a commit message or changelog) helps everyone track what's new or changed. During an incident is the worst time to be surprised that mycli get-user changed its output format — so manage updates deliberately.
By covering distribution, installation, and versioning, you ensure that when the pressure is on, everyone has access to the same reliable CLI tool, in the right version, wherever they need it.
Final Tip: Start Small and Iterate From Real Needs
You might be thinking, "This all sounds great, but where do I even start?" The best approach we found is to iterate from real needs. Don’t try to build a perfect, all-encompassing CLI from day one — that would be a lot of work, and you might guess wrong about what features matter. Instead, begin with the handful of curl patterns your team uses most often during on-call incidents or debugging sessions. For example, if you frequently have to check a user's profile and then disable something for that user, make those two commands first. Implement them carefully and make sure they truly simplify the task (get feedback from teammates), then add tests around them.
As those initial commands prove their worth (and believe me, your sleep-deprived self will thank you), you'll naturally discover the next most useful features or commands to add. Over time, your CLI can grow, but it will always be grounded in solving real, observed problems. In our case, we started with just two commands wrapping some particularly painful curl calls. Seeing how much time and hassle those saved, we gradually expanded the tool. Months later, it had a dozen commands, robust logging, and many of the conveniences described above — and it was saving us massive time in every incident.
Remember, even a small CLI can pay back massively in time saved and mistakes avoided. The next time you're dealing with a 2 AM emergency, you’ll be running a couple of straightforward mycli commands with confidence, instead of fiddling with curl and double-checking headers in a panic. For our team, that difference meant quicker recoveries and far fewer "Oops, I forgot to URL-encode that field" stories showing up in our Slack the next day.
In conclusion, building a custom CLI for your service is about making life easier for you and your colleagues. It's a bit of upfront effort that turns into peace of mind during stressful moments. So if you find yourself repeatedly executing complex curl commands or scripts while debugging, consider spending a little time to package that knowledge into a friendly CLI. Your future self (and your team) will thank you — and you might just get to catch a bit more sleep on your next on-call night. Happy debugging!
Opinions expressed by DZone contributors are their own.
Comments