Debian Codegraph @

examples/tiny/docs/transcript_2_tiny_interpreter.md

raw
1 Instruction:
2 ```
3 In tiny_parser.py, define number using ppc.number, not ppc.fnumber. This will convert numeric values to Python floats or ints at parse time, so no additional conversions are required in tiny_ast.py or tiny_engine.py.
4 ```
5
6 Plan:
7 - Update `examples/tiny/tiny_parser.py` to use `pp.common.number` instead of `pp.common.fnumber` for the `number` literal.
8 - Verify that the parser still builds expressions correctly and that numeric tokens are converted to Python `int`/`float` at parse time.
9 - No changes required to `tiny_ast.py` or `tiny_engine.py` since runtime conversion becomes unnecessary.
10
11 Actions taken:
12 - Replaced `ppc.fnumber` with `ppc.number` and kept the token name as `"Number"`.
13 - Reviewed term and expression rules to ensure they continue to accept the `number` literal without further changes.
14 - Left engine and AST modules untouched, as requested.
15
16 ---
17
18 #### Entry 19 — User Instruction
19 Instruction:
20 ```
21 Open a new file in tests/test_tiny_ast_nodes.py, with unit tests for the Declaration, Assignment, and Repeat statements. Use pytest form, not unittest.
22 ```
23
24 Plan:
25 - Create `tests/test_tiny_ast_nodes.py` with pytest-based tests covering:
26 - Declaration statements with optional initializers (verify values via output).
27 - Assignment statements updating an existing variable (including expression evaluation).
28 - Repeat-until loop printing a message N times while decrementing a counter. ✓
29 - Each test will:
30 - Parse a small Tiny program with `parse_tiny`.
31 - Build the `MainDeclNode` via `TinyNode.from_statement_type` and execute with `TinyEngine`.
32 - Capture stdout using `capsys` and assert expected text and return value. ✓
33 - Ensure tests rely on existing interpreter behavior without modifying engine/AST logic. ✓
34
35 Actions taken:
36 - Added new file `tests/test_tiny_ast_nodes.py` with three tests: declarations with initializers, assignment update, and a repeat-until loop that prints a fixed number of lines.
37 - Tests build and execute the main AST node and assert both stdout and the `return` value.
38
39 #### Entry 18 — User Instruction
40 Instruction:
41 ```
42 Implement DeclStmtNode (which may also include an assignment portion).
43 ```
44
45 Plan:
46 - Review the grammar for `decl_stmt` to confirm fields: `datatype` and a list `decls`, where each decl group has `name` and optional `init` expression. ✓
47 - Implement `DeclStmtNode.execute(engine)` in `examples/tiny/tiny_ast.py`:
48 - Determine `dtype = str(self.parsed.datatype)`.
49 - Iterate `self.parsed.decls`; for each decl, evaluate `init` with `engine.eval_expr` if present, then call `engine.declare_var(name, dtype, init_val)`. ✓
50 - Keep return value `None` and avoid adding new scope (declarations occur in current frame). ✓
51 - Update this transcript entry with actions taken. ✓
52
53 Actions taken:
54 - Implemented `DeclStmtNode.execute` in `examples/tiny/tiny_ast.py` to declare one or more variables with optional initializers, using the engine for expression evaluation and declaration into the current frame.
55 - No changes to other nodes or engine were required.
56
57 #### Entry 14 — User Instruction
58 Instruction:
59 ```
60 Add an is_implemented classmethod to TinyNode to detect if an execute() method has been defined (return False if cls.execute is TinyNode.execute). This will help us incrementally implement AST classes.
61 ```
62
63 Plan:
64 - Add a `@classmethod is_implemented() -> bool` to `TinyNode` in `examples/tiny/tiny_ast.py`.
65 - Implement it to compare `cls.execute` against `TinyNode.execute` and return `True` only when overridden.
66 - Leave all other code unchanged.
67 - Append this entry to the running transcript.
68
69 Actions taken:
70 - Edited `examples/tiny/tiny_ast.py` to add `TinyNode.is_implemented` that returns `cls.execute is not TinyNode.execute`.
71 - No other modules required changes.
72
73 #### Entry 10 — User Instruction
74 Instruction:
75 ```
76 Add a do-nothing execute() method to the base TinyNode class, which takes a TinyEngine as an argument. Then implement execute() for MainDeclNode, WriteStmtNode and ReturnStmtNode.
77 ```
78
79 Plan:
80 - Add a no-op `execute(engine: TinyEngine) -> object | None` method to the `TinyNode` base class.
81 - Implement `execute` for `MainDeclNode` to:
82 - Push a new frame on the `TinyEngine`.
83 - Iterate the statements in `main.body.stmts`, wrapping them into `TinyNode` subclasses via `from_statement_type` and calling `execute`.
84 - Stop and return when a `ReturnStmtNode` is encountered (or when a child returns a non-None value).
85 - Pop the frame and return the captured value.
86 - Implement `execute` for `WriteStmtNode` to evaluate its `expr` via the engine and write the stringified result, or output a newline for `write endl;`.
87 - Implement `execute` for `ReturnStmtNode` to evaluate and return its expression value.
88 - Keep other node classes skeletal; no CLI changes required.
89
90 Actions taken:
91 - Updated `examples/tiny/tiny_ast.py`:
92 - Added a default no-op `execute(self, engine)` method to `TinyNode`.
93 - Implemented `execute` in `MainDeclNode` to push/pop a frame, iterate and execute statements, and stop on return.
94 - Implemented `execute` in `WriteStmtNode` to write evaluated expressions or newline for `endl`.
95 - Implemented `execute` in `ReturnStmtNode` to evaluate and return its expression value.
96 - Left other nodes skeletal and made no changes to `tiny_run.py` or `tiny_engine.py`.
97
98 ---
99
100 #### Entry 20 — User Instruction
101 Instruction:
102 ```
103 Implement ReadStmtNode that will execute "read x" by using a Python input(f"x? ") statement. Add unit test for this class to test_tiny_ast_nodes.py
104 ```
105
106 Plan:
107 - Implement `ReadStmtNode.execute(engine)` in `examples/tiny/tiny_ast.py`:
108 - Extract the identifier name from `self.parsed.var`.
109 - Prompt and read input using Python `input(f"{name}? ")`.
110 - Assign the entered string to the variable via `engine.assign_var(name, value)` so the engine applies type-aware coercion based on any prior declaration. ✓
111 - Add a pytest in `tests/test_tiny_ast_nodes.py`:
112 - Program: `int x; read x; write x; newline; return 0;`.
113 - Monkeypatch `builtins.input` to return `"17"`.
114 - Execute main via `TinyEngine`, capture stdout, and assert output equals `"x? 17\n"` and return value is `0`. ✓
115 - Keep other files unchanged; do not implement broader function-call input handling. ✓
116
117 Actions taken:
118 - Edited `examples/tiny/tiny_ast.py` to add `ReadStmtNode.execute` that prompts with `input(f"{var}? ")` and assigns the value using `engine.assign_var`.
119 - Updated `tests/test_tiny_ast_nodes.py` to add `test_read_statement_prompts_and_assigns`, monkeypatching `builtins.input` to simulate user entry and asserting the combined prompt and output.
120 - No changes required to the parser or engine; existing coercion logic converts the string `"17"` to an int for a previously-declared `int x`.
121
122 #### Entry 11 — User Instruction
123 Instruction:
124 ```
125 Modify TinyEngine to also keep a dict of globals, for variables that can be accessed from any TinyFrame. Store any variables or functions defined at the program level in the dict of globals.
126 ```
127
128 Plan:
129 - Extend `TinyEngine` with a dedicated program-level globals store and a functions registry.
130 - Keep locals in stack `TinyFrame`s as before; the current frame remains the last item in the stack.
131 - Update lookup/assignment logic to search local frames first, then fall back to globals.
132 - Add explicit APIs for manipulating globals (`declare_global_var`, `assign_global_var`, `get_var` to read from globals if not found locally) and for function registration (`register_function`, `get_function`).
133 - Ensure `eval_expr` also consults globals when resolving identifiers.
134 - No parser/CLI changes required in this step; future steps can populate globals/functions based on program-level constructs.
135
136 Actions taken:
137 - Updated `examples/tiny/tiny_engine.py`:
138 - Added `_globals: TinyFrame` and `_functions: dict[str, object]`.
139 - Frame stack `_frames` now holds only local frames; globals are separate and always present.
140 - Implemented program-level registry methods `register_function` and `get_function`.
141 - Implemented globals API: `declare_global_var`, `assign_global_var`; updated `get_var`, `_find_frame_for_var`, and `assign_var` to consult and write to globals when appropriate.
142 - Adjusted `current_frame` to guard against use when no local frame exists.
143 - Updated `eval_expr` identifier resolution to check locals first, then globals.
144 - No changes to `tiny_ast.py` or `tiny_run.py` were required for this step.
145
146 ---
147
148 #### Entry 12 — User Instruction
149 Instruction:
150 ```
151 Change lines 71-77 in tiny_run.py to:
152 - create a new TinyEngine
153 - build nodes for the nodes in parsed
154 - save all top-level functions (including main()) into the TinyEngine functions, and any top-level variables in to the TinyEngine globals
155 - extract the AST node for the top-level "main" function, and call its execute() method passing in the TinyEngine object
156 ```
157
158 Plan:
159 - Modify `examples/tiny/tiny_run.py` to replace the placeholder post-parse logic.
160 - Instantiate a `TinyEngine`.
161 - Register all top-level function definitions from `parsed.program.functions` into the engine via `register_function`.
162 - If present, load any top-level globals into the engine using `declare_global_var` (guarded, since the current grammar may not expose them).
163 - Build the AST node for `program.main` using `_build_nodes`, register it as `"main"`, and execute it by calling `execute(engine)`.
164 - Keep CLI return code and error handling intact.
165
166 Actions taken:
167 - Updated `examples/tiny/tiny_run.py` lines after parsing to:
168 - Create `engine = TinyEngine()`.
169 - Iterate `program.functions` to `register_function(name, fdef)`.
170 - Best-effort import of any `program.globals` into engine globals when present.
171 - Build `main_node = _build_nodes(program.main)`, register it as `"main"`, and execute it when it is a `TinyNode`.
172 - Left parsing and `--dump` behavior unchanged.
173
174 ---
175
176 #### Entry 13 — User Instruction
177 Instruction:
178 ```
179 At the end of WriteStmtNode, add "engine.output_text()". Change TInyEngine's output_text() method to print("".join(self._out)), followed by self._out.clear()
180 ```
181
182 Plan:
183 - Update `WriteStmtNode.execute` in `examples/tiny/tiny_ast.py` to call `engine.output_text()` after writing output or newline, to flush the buffer. *
184 - Modify `TinyEngine.output_text` in `examples/tiny/tiny_engine.py` to print the buffered output and clear the buffer (no return value). *
185 - Verify that no other code depends on `output_text()` returning a string. ✓
186 - Update `examples/tiny/tiny_interpreter_transcript.md` with this entry documenting instruction, plan, and actions. *
187 - Quick sanity check by running `hello.tiny` manually (not executed here).
188
189 Actions taken:
190 - Edited `examples/tiny/tiny_ast.py`: in `WriteStmtNode.execute`, added a call to `engine.output_text()` after `_write`/`_writeln` to flush output immediately.
191 - Edited `examples/tiny/tiny_engine.py`: changed `output_text` to `print("".join(self._out), end="")` and then `self._out.clear()`; updated signature to return `None`.
192 - Searched current usages; no part of the codebase relied on the previous return value of `output_text`.
193
194 #### Entry 15 — User Instruction
195 Instruction:
196 ```
197 Refactor MainDeclNode.execute() to remove the conversion from ParseResults to AST nodes. Instead, this conversion should be done in a build_contained_statements() method that is called from __init__() after calling super().__init__. build_contained_statements() will iterate over the statements in the parsed body, and create a list of statement nodes. Then execute() merely has to iterate over those nodes and call execute on each one. If those nodes themselves contain a body of statements, then building that AST node will in turn recursively build AST node lists - this will be the case for the main, function, repeat, and if statements.
198 ```
199
200 Plan:
201 - In `examples/tiny/tiny_ast.py`, update `MainDeclNode` to pre-build its contained statements:
202 - Add `__init__` that calls `super().__init__(parsed)` then `self.build_contained_statements()`.
203 - Implement `build_contained_statements()` to iterate `self.parsed.body.stmts`, resolve each via `TinyNode.from_statement_type`, and instantiate nodes into `self.statements: list[TinyNode]`.
204 - Refactor `MainDeclNode.execute()` to only iterate over `self.statements`, executing each node and stopping on `ReturnStmtNode` or non-None result; keep frame push/pop with `finally`.
205 - Keep type annotations in Python 3.12+ style; avoid import cycles by using forward ref for `TinyEngine`.
206 - Append this entry to the transcript.
207
208 Actions taken:
209 - Edited `examples/tiny/tiny_ast.py`:
210 - Added `__init__` and `build_contained_statements()` to `MainDeclNode` to construct `self.statements` at initialization.
211 - Simplified `MainDeclNode.execute()` to iterate over prebuilt `self.statements` and execute them, preserving early return behavior and frame management.
212 - No changes were required in other files.
213
214 ---
215
216 #### Entry 16 — User Instruction
217 Instruction:
218 ```
219 Implement AssignStmtNode.execute() method and RepeatStmtNode.build_contained_statements() and .execute() methods
220 ```
221
222 Plan:
223 - In `examples/tiny/tiny_ast.py`, implement `AssignStmtNode.execute(engine)` to evaluate the RHS using `engine.eval_expr(self.parsed.value)` and assign it to `self.parsed.target` via `engine.assign_var(...)`. ✓
224 - Implement `RepeatStmtNode` to pre-build its body statements:
225 - Add `__init__` that calls `super().__init__(parsed)` then `build_contained_statements()`. ✓
226 - Implement `build_contained_statements()` to iterate `self.parsed.body.stmts`, map each to a `TinyNode` subclass, and store in `self.statements: list[TinyNode]`. ✓
227 - Implement `RepeatStmtNode.execute(engine)` with do-while semantics:
228 - Execute all body statements in order; if any returns a non-None result or is a `ReturnStmtNode`, propagate that result immediately (to allow function returns inside loops). ✓
229 - After each body execution, evaluate `self.parsed.cond` with `engine.eval_expr`; break the loop when the condition is truthy. ✓
230 - Do not push/pop frames in `RepeatStmtNode` (no new scope). ✓
231 - Update the running transcript with this entry. ✓
232
233 Actions taken:
234 - Edited `examples/tiny/tiny_ast.py`:
235 - Implemented `AssignStmtNode.execute` to evaluate and assign the RHS to the target variable via the engine.
236 - Added `__init__`, `build_contained_statements`, and `execute` to `RepeatStmtNode` to pre-build its body and run it with repeat-until semantics, including early return propagation.
237 - Updated `examples/tiny/tiny_interpreter_transcript.md` with this plan and the actions performed.
238
239 ---
240
241 #### Entry 17 — User Instruction
242 Instruction:
243 ```
244 TinyEngine._to_number() must be modified:
245 - do not convert v to float if v is an int or float
246 - if v is a string, get the value of the variable v; return that value if int or float, otherwise raise TypeError
247 ```
248
249 Plan:
250 - Update `_to_number` in `examples/tiny/tiny_engine.py` to conform to the specified behavior:
251 - If `v` is `int` or `float`, return `v` unchanged. ✓
252 - If `v` is `str`, resolve it as a variable name using existing lookup (`get_var`). If the value is numeric, return it; otherwise raise `TypeError`. ✓
253 - For any other type, raise `TypeError`. ✓
254 - Ensure arithmetic and relational operations that use `_to_number` continue to work without further changes. ✓
255 - Append this entry to the transcript. ✓
256
257 Actions taken:
258 - Edited `examples/tiny/tiny_engine.py`:
259 - Rewrote `_to_number` to return `int | float` and to return numeric inputs unchanged, resolve strings as identifiers via `get_var`, and raise `TypeError` for non-numeric or unresolved identifiers.
260 - Removed previous generic `float(str(v))` coercion path to avoid unintended conversions.
261
262 ---
263
264 #### Entry 22 — User Instruction
265 Instruction:
266 ```
267 Implement function calls in TinyEngine.eval_expr
268 ```
269
270 Plan:
271 - Inspect how functions are parsed and registered: function definitions are stored as `ParseResults` with `decl` (name, return_type, parameters) and `body.stmts`, and they are registered into the engine in `tiny_run.py`. ✓
272 - Add a reusable function invocation path in the engine so both expression `func_call` and a future `CallStmtNode.execute` can use it: *
273 - In `eval_expr`, when encountering a `ParseResults` with `type == 'func_call'`, evaluate arguments left-to-right and delegate to a new `engine.call_function(name, args)` method. *
274 - Implement `call_function(name, args)` to look up the function, push a new frame, bind parameters by declaring them with their declared types and the coerced argument values, execute the body statements one by one (building each as a `TinyNode` and calling `execute(engine)`), return on `ReturnStmtNode` or any non-`None` result, and finally pop the frame. If no explicit return occurs, return the default value for the declared return type. *
275 - Provide a small helper `_build_stmt_node(stmt_group)` that maps a statement `ParseResults` group to its `TinyNode` subclass instance via `TinyNode.from_statement_type`. *
276 - Keep existing identifier/number/string handling and operator semantics intact. ✓
277 - Do not implement `CallStmtNode.execute` yet; ensure the engine API is ready to be reused there. ✓
278
279 Actions taken:
280 - Edited `examples/tiny/tiny_engine.py`:
281 - Imported `TinyNode` and `ReturnStmtNode` from `tiny_ast`.
282 - In `eval_expr`, added handling for `func_call` to evaluate argument expressions and call `self.call_function(name, arg_values)`.
283 - Implemented `_build_stmt_node` helper for converting statement groups into `TinyNode` instances.
284 - Added `call_function(name, args)` implementing function invocation: lookup, frame push, parameter binding with type coercion, sequential execution of body statements with early-return propagation, default return value when necessary, and frame pop in a `finally` block.
285 - No changes to the parser or AST node classes were required in this step; `CallStmtNode` can later call `engine.call_function(...)` to share this logic.
286
287 ---
288
289 #### Entry 23 — User Instruction
290 Instruction:
291 ```
292 Implement CallStmtNode.execute
293 ```
294
295 Plan:
296 - Confirm parser shape for `call_stmt`: it wraps a nested `function_call` group carrying `type == 'func_call'`, `name`, and optional `args`. ✓
297 - Implement `CallStmtNode.execute(engine)` in `examples/tiny/tiny_ast.py` to:
298 - Locate the nested `func_call` group.
299 - Evaluate each argument via `engine.eval_expr`.
300 - Invoke `engine.call_function(name, arg_values)` and ignore the return value (statement context). ✓
301 - Update this running transcript with instruction, plan, and actions. ✓
302
303 Actions taken:
304 - Implemented `CallStmtNode.execute` in `examples/tiny/tiny_ast.py`:
305 - Finds the inner `func_call` group, evaluates its `args` with `engine.eval_expr`, and calls `engine.call_function(name, arg_values)`.
306 - Ignores any returned value and returns `None` as expected for a statement.
307 - Left tests unchanged; future tests can exercise statement-style calls using existing function support in the engine.
308
309 ---
310
311 #### Entry 24 — User Instruction
312 Instruction:
313 ```
314 Remove is_implemented from TinyNode, was never used
315 ```
316
317 Plan:
318 - Remove the `is_implemented` classmethod from `examples/tiny/tiny_ast.py`'s `TinyNode` since it is unused. ✓
319 - Search codebase references to `is_implemented` to confirm no usages (recent context indicates none). ✓
320 - Update this transcript with the instruction, plan, and actions. ✓
321
322 Actions taken:
323 - Edited `examples/tiny/tiny_ast.py` to delete the entire `TinyNode.is_implemented` classmethod.
324 - Verified recent files and tests do not reference `is_implemented`; no further changes required.