AI LLM Fact Checking the June 2024 Debate
I am fact checking the debate. https://x.com/BerlinBrownMech ,,,
Posts
So my LLM LLama2 training did work
So it took about 4 hours on my mac without GPU improvements but this command ran for about 4 hours. python3 train.py --compile=False --vocab_source=custom --vocab_size=4096 And then I could run against the llama2.c code. ./run ./out/model.bin I got this gibberish: "pres es eluted эargared copy É Int beforepperunch KarologfromwayClassistoryork ochidentAr}^ Allet Com easgoogleiden targetegaoman essпиgesscript non behS commandasesasesба amb before pervecnode agcolorkoeln conf Ma Setrat Textema governiowhere ##скогоchange.) respectankön knowPar namesiones неander für enrid muool medcia depBalityви rangehelова () del options ### voando](arget Thereise und descri L`,incless++readble oldredULLockabelutesphaires says буClientIC});viroo test only ser"
Updated on research llm and llama2
So here are some more keywords and updates to learning llm and the like. So I running the code from here. https://github.com/karpathy/llama2.c Mainly the training piece. So these ran fine: python tinystories.py download python tinystories.py pretokenize But had issue with the time to run this on a mac machine without cuda installed. This took several days to run: python3 train.py --compile=False --vocab_source=custom --vocab_size=4096 Overriding: compile = False Overriding: vocab_source = custom Overriding: vocab_size = 4096 tokens per iteration will be: 131,072 breaks down as: 4 grad accum steps * 1 processes * 128 batch size * 256 max seq len Initializing a new model from scratch num decayed parameter tensors: 43, with 7,151,616 parameters num non-decayed parameter tensors: 13, with 3,744 parameters using fused AdamW: False Created a PretokDataset with rng seed 42 Created a PretokDataset with rng seed 42 97 | loss 6.4552 | lr 4.850000e-05 | 1962367.71ms | mfu 0.01% 98 | loss 6.4...
Ode to Lisp - Blog Entry
I have never had much affinity for writing in natural language. I grew up enjoying programming, and that passion has continued for over 20 years. However, I felt compelled to write this blog entry because it helped me rethink and reframe my perspective on a particular piece of code. I'll be exploring various concepts, starting with one of the most fundamental data structures in computer science: the linked list. Instead of considering Lisp code as a mere "listing" of elements, try to envision it as a linked list of elements. https://berlinbrowndev.blogspot.com/2008/07/simple-lisp-implementation-in-java-ode.html
On Wolfram Alpha and Cellular Automata
When most computer users upload a profile image from their desktop to Facebook, they rarely consider the fundamental binary math rules underpinning digital devices. We know that 4 gigabytes of RAM is more memory than 512 megabytes, but we don't visualize the logic chips involved in an XOR $0x100, EAX operation for a 32-bit CISC processor. Software developers must consider memory management and how a computer's operating system loads their programs into memory. However, they typically don't think about VHDL logic circuit designs, data paths, arithmetic logic units, or the millions of transistors comprising a modern CPU. These low-level details are intentionally abstracted away from the user application developer. While modern CPUs have evolved dramatically over the past decade, early digital computing relied on simple Boolean operations. These fundamental rules were combined and replicated to load programs into memory and execute them. The principles controlling most digita...
On Unit Testing Updates
I have been reading about four or five posts a day on unit testing, an obsession that has persisted for a long time. I've moved beyond the technical and practical considerations of unit testing frameworks and have finished debating whether to use JUnit, Mockito, or Karma. Now, I am more intrigued by the psychology of unit testing—who engages in it, who enjoys it, and who dislikes it? Unit testing is one of those concepts that are easy to learn but hard to master. For example, many people play chess when they are young but remain poor players throughout their lives. I am part of that majority. I have never dedicated hours to playing chess or attempting to master it. I don't recognize common patterns or have a well-developed endgame. I simply play with a basic understanding of the rules. Following good unit testing practices in your software development team is a lot like playing chess: easy to learn but difficult to master. However, there are significant differences—chess is a ...