Nugget 24: Token Economy Management — Comments Are Not Free

Expanding, just a little, on Nugget 23, where the fundamental idea is "Token Economy Management" (TEM), I found something literally disturbing: The Cost of Good Habits → Comments.

Computer science professors have always told us that someone else will need to read our code and that, after we have departed, other software engineers and developers will look at what we did. Let them know with comments.

For the following test, the file IEnumerator from System.Collections was selected. TEM was analyzed. These are the findings:


==================================================

Traditional Comments vs. No Comments

==================================================

Commented Function Token Count:    295

Non-Commented Token Count:            65


Tokens Saved:                                  230

Context Window Reduction:          77.97%

==================================================


Remember, the LLM cannot filter comments. These comments must still be processed. Every one of those tokens enters the context window. Every one of those tokens must be embedded, attended to, and reasoned about.

Perhaps, if we were working only with Web APIs, we could offload documentation to metadata schemas such as OpenAPI (Swagger). This would alleviate the problem, allowing the inline XML comments associated with controllers and endpoints to be removed while letting the tooling generate the OpenAPI specification.

The message is not that comments are bad.

If the message you perceive after reading this note is that "we should not write comments," I apologize; that is not what I mean.

I am calling attention to the LLM processing burden because comments are no longer free.

For decades, comments were essentially a storage concern. In the age of LLMs, they are also a computational concern. Every token must be processed, and that processing carries a cost. TEM simply provides a way to measure that cost and make it visible.

Comments

Post a Comment

Popular posts from this blog

Nugget 11: Adding Filtering to a Blazor QuickGrid

Image
  The best way to add filtering to a Blazor QuickGrid is by creating an expression tree representing the predicate of a LINQ Lambda Expression Where clause. In  Nugget 9  , a simple Blazor app that supports CRUDE operations was created using scaffolding and Entity Framework.  Because the app loads a collection of items from the database and index component containing a QuickGrid component was created to display the collection of items. The above code represents the Index.razor component, which was modified after it was created.  Directives are omitted for simplicity. The component filters the collection of JobPosts by two model properties: JobPost.ApplicationDeclined , and JobPost.JobType using a checkbox and an input field.  The input field is defined in the ColumnOptions for the JobType PropertyColumn  and is bound to a component string variable, jobTypeSearch . Because this variable is bound to the input field, the variable is updated whene...
Read more

Nugget 9: Building a JobBank App using Blazor. A Nugget with a Story.

Image
One of my daughters is about to graduate in a few days, and she has a contract lined up already as a Game Developer! As a happy Dad, I am very proud of her accomplishments.   " Dad, how do I keep track of the employment opportunities I have applied to? " Good question; you need to keep track of who you send your resume to. Your email application outbox doesn't work well because some job applications are submitted online, and most of the time, the platform that handles applications doesn't send you a thank-you email. Plus, you could see the same job elsewhere and apply again by mistake, which is detrimental to us. You could create an Excel document and use it as a job submission database. You could search, filter, and add notes and comments after a phone call or interview. You could also make a record of any topics you might need to review before the initial contact with the potential employer based on the job description. This is practical, but not as fun. " How ...
Read more

Nugget 14: Connection to MongoDB Atlas - SSL Alert 80

Image
  I'm using MongoDB Atlas (1) to host my NoSQL database. Creating an application with the MERN stack was very straightforward. However, after a few hours, when I tried to run the same application, Atlas displayed an SSL error message. How is this possible? What changed overnight? Something changed overnight, of course! When creating the cluster in Atlas, my IP address was configured (2), and by default, it expired after 6 hours. The error message was a bit misleading, although the problem was discovered after some investigation. Either re-whitelist your IP address or remove and re-add it to fix the error. I had a harder time finding the solution in Atlas, so I'm writing it down. Log in to Atlas, go to Network Access, and fix it there.  There is also an option to be able to access the app from any IP.  I prefer to restrict the IP. After the correction, it worked as expected. The Atlas MongoDB database is impressive (3, 4).   References & Notes MongoDB Atlas ...
Read more