About

Tools and tests built with AI, and what they measured

Protocol AI Lab publishes small tools and hands-on tests made with AI coding tools such as Claude Code and Codex. Every post is built around something that was made or measured here: a tool you can open in your browser, a test with its results table, or a demo with the checklist it was checked against.

The AI tools do most of the hands-on work. They write the code, the tests and the first drafts. I decide what is published. What you read is the result of actually running that work, including the parts that did not hold up and the limits of each test.

What you will find here

Telecom and protocol tools

This is the work closest to my own background in SIP signaling.

  • SIP Call Flow Visualizer. Paste a SIP log and get a ladder diagram, a list of problems such as missing ACKs, error responses, retransmissions and calls with no common codec, and a summary of the SDP offer and answer. The log is read in your browser and is not uploaded. Version 0.1.1, MIT license, 23 automated tests on synthetic traces. It has not yet been checked on real field traces.
  • How the SIP tool was built, with the prompts that were used, the dead ends (including WordPress rewriting the tool’s script) and the measured timeline.

AI coding tool experiments

Small tests of AI models, of the rule files people give coding agents, and of ways to work with them. Each post states how many runs it had and links the files behind the results.

  • Opus 5.5 benchmark. Opus 5.5, Opus 5, Sonnet 5 and Haiku 4.5 on SIP tasks, a protocol invented for the test and long multi-step traces, with 288 graded calls on these four models over four rounds. The first round also graded 12 calls on two other models, Fable 5.1 and Opus 4.7. On the shorter tasks of the main rounds every model averaged 97% or more. Only the round of long tasks separated them: Opus 5.5 100%, Sonnet 5 99.5%, Opus 5 92.1%, Haiku 4.5 57.7%.
  • Project rule files in Codex. Sixteen Codex runs on two small tasks, a SIP parser fix and a caching request that gave no cache policy. Each task ran four times with a project AGENTS.md rule file (the Codex counterpart of CLAUDE.md) and four times without one. On both tasks the two groups ended the same way. All eight parser runs passed their own test suites, and a separate seven-case check found the same missed edge case once in each group. All eight cache runs stopped to ask for a policy and left the code unchanged.
  • LLM wiki portability test. Codex turned three conflicting notes into a small wiki. When only the wiki folder was copied, the answer stayed correct but 18 links to the sources broke. Keeping copies of the sources inside the wiki brought that to zero.

Calculators and demos, with what was measured

Calculators that run in your browser, and demos built on made-up data. The calculators and finance posts explain the arithmetic. They are not investment, tax or legal advice. The posts in this group are mostly in Korean.

How posts are made

  • Where topics come from: my own protocol work, new AI tool releases, problems described in public project listings, and questions noticed in news headlines.
  • Who does the work: AI coding tools, mainly Claude Code and Codex, write the code, tests, charts and first drafts. Each post states where AI tools were used in the work it describes.
  • Where the numbers come from: from running the code. When the data, scripts or source are published, the post links to them so you can check the numbers yourself.
  • News headlines: scripts collect them every day to help spot questions worth testing. This site does not republish news articles or publish news summaries.
  • Publishing: drafts stay as WordPress drafts and nothing is published automatically. A post goes live only after I have read it and approved it.

About the author

I am J. Kim, a telecom protocol engineer with more than 20 years of production experience. Most of that time has been spent on carrier-grade systems:

  • SIP, INAP and TCAP signaling development
  • TCP/IP-based integration processes between multi-vendor equipment
  • Server applications in C, C++, Java and Python
  • Red Hat Enterprise Linux environments with strict uptime requirements

That background shapes how I test AI tools. I care less about how a result looks and more about whether it survives real constraints: memory safety, failure handling, protocol correctness and long-running reliability. It is also why one of the tools here reads SIP traces, and why several of the AI tests use SIP tasks.

Editorial and AI policy

  • AI use is disclosed. This is a site about building with AI, so AI tools are used in the projects and also write the first drafts of posts. Posts and tool pages state where AI tools were used in the work they describe.
  • Everything shown was actually run. Code, output, tables and screenshots come from the real build or test, not from generated examples.
  • Limits are stated. Small samples, synthetic data and anything that was not tested are named in the post.
  • No copied content. I do not republish news articles or other people’s work. Sources are linked when referenced.
  • Corrections. If you find a mistake, tell me through the Contact page and I will fix it and note the change.
  • Removed posts. In October 2026, a series of automated posts that counted words in news headlines was taken offline. Those posts were mainly about headline counts, not the tools and tests this site is for.

AI disclosure for this page: Claude Code drafted this version of the About page from the site’s project files and published posts. J. Kim read and approved this revision on October 8, 2026.

Get in touch

Questions, project ideas or corrections are welcome. Please use the Contact page.

한국어 소개

Protocol AI Lab은 통신 프로토콜 엔지니어 J. Kim이 Claude Code·Codex 같은 AI 코딩 도구로 만든 도구와, 실제로 돌려 본 시험의 결과를 올리는 곳입니다. SIP 로그를 호 흐름 그림으로 바꾸는 도구, AI 모델과 규칙 파일을 같은 과제로 비교한 시험, 예금·적금과 채권 계산기, 가상 데이터로 만든 업무 데모가 있고, 글마다 실제로 잰 값과 시험하지 못한 부분을 함께 적습니다. 코드·시험·그래프·글 초안은 대부분 AI 도구가 쓰고, 무엇을 공개할지는 J. Kim이 정합니다. 뉴스 제목은 스크립트가 매일 자동으로 모아 시험해 볼 질문을 찾는 데 쓰며, 뉴스 기사나 요약은 싣지 않습니다. 자동으로 공개되는 글은 없고, J. Kim이 읽고 승인한 글만 공개합니다. 계산기와 금융 글은 계산 방법을 이해하기 위한 자료이며 투자 권유가 아닙니다.

Latest posts

Last updated: October 8, 2026

Scroll to Top