TinyCoreOS¶

Applies to version: TinyCoreOS Pre-Alpha 5 2.0

A graphical operating system built from scratch in C++20 and x86-64 assembly.
Current milestone: M44 - Preferences System
Next: M45 - Shell Window (CLI)
Prebuilt images for VMWare: https://tinycoreos.com/build/.

Video: Development trailers and demo clips.

Trailers: YouTube Trailer 1 | YouTube Trailer 2 | YouTube Trailer 3

About me: Adrian Pucek. Contact: [enable JavaScript]

Index¶

Build and run

• Prerequisites
• Quick start
• Make targets

Daily usage

• CoreShell
• Not Commander (nc)
• Networking
• HTTP Server (httpd)
• FTP Server (ftpd)
• Disk workflow (format / mount)

Reference

• Project layout
• Architecture overview
• Supported drivers
• Toolchain
• Milestones
• PHP migration plan
• Roadmap
• Historical notes

First steps (5 minutes)¶

After make run and boot/login in TinyCoreOS:

help
ipconfig
resolve github.com
ls /disk
date
time
nc

Tip: if DNS/network is not up yet, run netinit and retry resolve.

Latest CoreDesk updates¶

• Screen menu TinyCoreOS → Run Program now opens an in-window run dialog (no external run_dialog.elf dependency).
• The run dialog executes shell commands with arguments (PATH lookup + execve), e.g. notepad.elf /tmp/readme.txt.
• Power-style actions now use confirmation dialogs: Quit CoreDesk, Restart, Shut Down.
• In Apps → Tools, a new Terminal entry launches sh (fallback: busybox sh).
• Apps → System no longer contains Power (power actions remain in the TinyCoreOS menu).

Latest CoreShell updates¶

• USB stage-2 detection (wait + probe + MSC/HID) now runs asynchronously after STI; boot reaches login: without waiting on the USB settle delay.
• During login, USB stage-2 logs are batched and printed as one block, then login: / Password: is repainted once (no login: [INFO] ... interleave).
• Login caret repaint was hardened for async boot logs to avoid stray _ artifacts at line start.
• find <path> [name] now supports wildcard match (*, ?) in name; when no wildcard is present it keeps substring matching.
• copy / cp now support wildcard source expansion (*, ?) in the last path component.
• grep path operands now support wildcards; added -s (sort output) and -u (unique output).
• cat now supports path wildcards plus -s/-u line output options.
• tail now supports wildcard paths in non-follow mode; tail -f requires exactly one resolved file.
• zip input arguments now support file glob expansion (*, ?).
• rm now supports wildcard operands (*, ?) with existing -r/-f/-v behavior.
• cat remains interruptible with Ctrl+C; tail -f remains a live append view until Ctrl+C.
• ver prints product codename + version from release macros (TCOS_PRODUCT_CODENAME, TCOS_PRODUCT_VERSION).

Milestones¶

Prerequisites¶

The toolchain is installed natively under MSYS2 - no Docker required.
See msys2-env.sh for the one-time setup script.

Quick start¶

# 1. Enter MSYS2 and load the toolchain path (once per session)
source ./msys2-env.sh

# 2. Download Limine bootloader binaries (once)
make limine-get

# 3. Download Monocypher crypto library (once, required for SSH)
make monocypher-get

# 4. Build kernel ELF + bootable ISO
make

# 5. Run in QEMU  (disk images created automatically)
make run

VMware Workstation (recommended VM profile)¶

TinyCoreOS works in VMware Workstation when the VM uses devices covered by current drivers.

• Firmware: BIOS or UEFI
• Disk controller: SATA (AHCI) or NVMe
• Network adapter: Intel E1000 / E1000E (recommended), or AMD PCnet
• Audio: HD Audio (Azalia) or AC97
• Graphics: VMware SVGA (framebuffer path works)

Notes:
- VMXNET3 is visible in lspci, but not yet supported by the network stack.
- Legacy SCSI controllers (BusLogic / LSI Logic Parallel) are not supported yet - use SATA/NVMe.
- VMware AHCI may expose many implemented-but-empty SATA ports; TinyCoreOS now suppresses those empty-port lines in boot output.

VMware debug logging setup (qemu-debug equivalent)¶

Use a virtual serial port mapped to a host file. This gives you a persistent boot/runtime log similar to QEMU -serial.

1. Power off the VM.
2. Open VM Settings -> Add... -> Serial Port.
3. Select Output to file and choose a host path, e.g. C:\temp\tinycoreos-vmware-serial.log.
4. Enable Connect at power on.
5. Boot the VM and inspect the log file on the host.

Notes:
- TinyCoreOS writes boot/kernel logs to COM1, so this captures early boot too.
- VMware also writes hypervisor/device diagnostics to vmware.log in the VM directory.
- If the serial log file stays empty, verify COM1 is present and connected.

PowerShell live tail (host):

Get-Content "C:\temp\tinycoreos-vmware-serial.log" -Wait

Debug targets¶

make debug          # QEMU paused, GDB on port 1234
make debug-boot     # QEMU with -no-reboot -d int,cpu_reset
make serial         # QEMU output to stdout, no window

Supported drivers¶

TinyCoreOS currently targets virtualized hardware profiles (QEMU / VirtualBox / VMware Workstation).

Network¶

• Intel E1000 family (e1000): 8086:100e, 8086:100f, 8086:10d3
• AMD PCnet (pcnet): 1022:2000, 1022:2001
• DHCP, DNS, TCP/UDP stack, loopback (127.0.0.1) are supported on top of these NICs.

Not supported yet:
- VMware VMXNET3 (15ad:07b0)
- VirtIO-Net

Storage¶

• ATA PIO (ata)
• AHCI SATA (ahci)
• NVMe (nvme)
• USB Mass Storage (SCSI transparent subset used by common flash drives)

Not supported yet:
- Legacy SCSI VM controllers (BusLogic / LSI Parallel)
- VirtIO-Blk / VirtIO-SCSI drivers

Audio¶

• Intel AC97 (8086:2415) via ac97
• Intel HD Audio / Azalia (hda)

Display and input¶

• Framebuffer console + desktop compositor path
• PS/2 keyboard and mouse
• USB keyboard (HID boot protocol)
• VMware SVGA works via framebuffer path (no dedicated SVGA II acceleration driver yet)

PCI ID -> driver module¶

Recommended VM profiles¶

• QEMU: e1000 + AHCI/NVMe + AC97 or HDA
• VirtualBox: Intel PRO/1000 (e1000) or PCnet + AHCI + HDA/AC97
• VMware Workstation: E1000/E1000E + SATA/NVMe + HD Audio

SMP diagnostics (dev)¶

When smpap is enabled in iso/limine.cfg, use these runtime checks:

cat /proc/smp_sched
smpkick 5
smpkick ap 1 5
cat /proc/smp_sched

Interpretation:
- resched_ipi_sent should increase after smpkick.
- resched_ipi_hits / resched_ipi_hits_cpuN confirm IPI delivery/handling per CPU.
- schedule_attempts / schedule_switches track scheduler activity under SMP tests.
- syscall_bkl_taken / syscall_bkl_bypassed show how much syscall traffic still goes
through the global BKL versus lock-free whitelist paths.
- syscall_bkl_taken_* highlights the hottest still-locked syscall families
(read, ioctl, futex, openat, close, wm_poll_event, wm_flush,
write, wm_create/destroy/map_fb, clipboard, readv/writev,
wm_get_size/wm_set_cursor/wm_move, file metadata paths (open/stat/fstat/lstat/getdents/access),
plus memory/time/path buckets (brk/mmap/munmap/clock_gettime/getcwd),
and additional scheduler/signal-fd buckets (poll/select/fcntl/rt_sigprocmask/set_tid_addr/waitpid),
plus process/UI text buckets (execve/clone/fork/kill/wm_set_title/font_draw_str/font_draw_to/shellcore_exec),
plus fd/plumbing buckets (lseek/pread64/pwrite64/dup/dup2/dup3/pipe/pipe2),
plus socket/net buckets (socket/connect/accept/accept4/sendto/recvfrom/bind/listen/shutdown
and net_dns_resolve/net_tcp_connect/net_tcp_connected/net_tcp_send/net_tcp_recv/net_tcp_close/net_poll),
plus WM/fclip extension buckets (wm_set_style/wm_set_desktop/wm_get_desktop/wm_win_desktop/wm_win_sticky
wm_set_mode/wm_set_cursor_bmp/wm_cursor_frame/fclip_begin/fclip_add/fclip_count/fclip_op/fclip_get/fclip_clear/fclip_gen),
plus WM menu/popup buckets (wm_menu_set/wm_menu_poll/wm_popup_show/wm_popup_poll/wm_get_focused/wm_raise),
plus WM/control buckets (wm_list_windows/power_off/reboot),
and other).
- syscall_bkl_taken_read_stdin / syscall_bkl_taken_read_fd and
syscall_bkl_taken_write_console / syscall_bkl_taken_write_fd split
read/write into fallback TTY/console paths versus fd-backed file paths.
- syscall_bkl_taken_waitpid_nohang / syscall_bkl_taken_waitpid_blocking
split wait family pressure into polling (WNOHANG) versus blocking waits.
- syscall_bkl_taken_mmap_anon / syscall_bkl_taken_mmap_file /
syscall_bkl_taken_mmap_fb0 split mmap pressure into anonymous, generic
file-backed, and framebuffer (/dev/fb0) mappings.
- syscall_bypass_* fields split lock-free traffic by syscall family
(sched_yield, nanosleep, socket name queries, kill(0,0), plus hot read-only
paths: getpid, gettid, getppid, getpgrp, getuid, geteuid,
get_ticks, brk, mmap(MAP_ANONYMOUS), waitpid(WNOHANG), fd-backed read/write, access, close, open (safe read-only subset),
wm_get_screen_size, wm_set_cursor, wm_set_style, wm_win_sticky, wm_get_focused, wm_menu_poll, wm_popup_poll, wm_list_windows,
wm_flush, wm_poll_event,
getdents,
font_measure, font_draw_to, clipboard_count, clipboard_gen, and other).

Current SMP checkpoint (incremental, stable):
- AP bring-up (smpap) is stable for 2 CPU QEMU runs.
- Reschedule IPI (smpkick) is wired and observable via /proc/smp_sched.
- Syscall path uses a conservative, argument-gated BKL bypass policy.
- sched_yield(24) and nanosleep(35) are now lock-free in syscall entry
(scheduler still applies its own SMP safety path internally).
- wm_poll_event(504) is now lock-free at syscall entry, backed by dedicated
WM internal locking for wid/event-ring critical sections.
- WM bridge now has dedicated internal locking for wid-table and event-ring
critical sections (prep step for safely reducing BKL pressure on WM poll path).
- FdTable operations (alloc/get/release/replace) are now internally
protected by a per-table spinlock as groundwork for future VFS-path BKL reductions.
- vfs::File now includes a dedicated per-file spinlock and atomic refcount
helpers, with vfs::read/write/readdir/ftruncate serialized per open-file instance.
- vfs::pread/pwrite now provide positional I/O with lock-scoped offset restore,
and mmap file-backed copying uses the same per-file lock domain.
- read(0) keyboard fallback and write(1/2) direct-console fallback remain under
BKL, but fd-backed read/write paths now bypass BKL and are tracked via
syscall_bypass_read_fd / syscall_bypass_write_fd.
- waitpid(61) now bypasses BKL for WNOHANG probes only; blocking waits stay
under BKL and telemetry exposes both sides via
syscall_bkl_taken_waitpid_nohang / syscall_bkl_taken_waitpid_blocking
and syscall_bypass_waitpid_nohang.
- wm_set_cursor(520) now bypasses BKL and is tracked via
syscall_bypass_wm_set_cursor; cursor mode updates use atomic loads/stores
in WM to avoid races with compositor-side hover updates.
- wm_set_style(530) and wm_win_sticky(534) now bypass BKL, backed by
existing wid-table locking in WM and tracked via
syscall_bypass_wm_set_style / syscall_bypass_wm_win_sticky.
- brk(12) now bypasses BKL; heap growth still synchronizes through PMM/VMM
internal locks, and lock-free traffic is tracked via syscall_bypass_brk.
- mmap(9) now bypasses BKL for MAP_ANONYMOUS mappings; file-backed mmap
paths remain under BKL and are split into mmap_file / mmap_fb0 telemetry.
- A dedicated VFS gate lock now protects syscall-level VFS metadata paths
(open/close/stat/lstat/fstat/access/getdents/openat) and enables lock-free
getdents dispatch without relying on the global BKL.
- close(3) now runs lock-free at syscall entry, guarded by the same VFS gate
+ per-task FdTable locking.
- open(2) now bypasses BKL only for a safe subset (O_RDONLY without
O_CREAT/O_TRUNC), keeping mutating opens under BKL.
- New telemetry counters help guide the next whitelist steps with runtime data.
- syscall_bkl_taken_other is now partially decomposed into dedicated buckets
(brk, mmap, munmap, clock_gettime, getcwd, wm_move) to narrow the
next lock-free candidate with less guesswork.
- Decomposition was extended with poll/select/fcntl/rt_sigprocmask/set_tid_addr/waitpid
to identify high-frequency non-VFS pressure still hidden under other.
- Decomposition was further extended with execve/clone/fork/kill/wm_set_title/font_draw_str/font_draw_to/shellcore_exec
to expose process-launch/UI-rendering overhead currently folded into other.
- Decomposition now also covers lseek/pread64/pwrite64/dup/dup2/dup3/pipe/pipe2
to isolate file-offset/descriptor plumbing pressure from residual other.
- Decomposition now also covers socket/net syscall families
(socket/connect/accept/accept4/sendto/recvfrom/bind/listen/shutdown
and net_dns_resolve/net_tcp_connect/net_tcp_connected/net_tcp_send/net_tcp_recv/net_tcp_close/net_poll)
to isolate network-path pressure from residual other.
- Decomposition now also covers WM extension (530-537) and file-clipboard (550-556)
syscall families to isolate desktop-control/file-clipboard pressure from residual other.
- Clipboard history/file-clipboard paths now use dedicated internal locking in WM,
enabling safe lock-free bypass for read-only clipboard_count/clipboard_gen.
- font_draw_to(516) now bypasses BKL (pure draw into caller-provided pixel
buffer), with dedicated syscall_bypass_font_draw_to telemetry.
- wm_get_focused(542) now bypasses BKL (read-only focused-window query),
backed by WM wid-table locking and dedicated syscall_bypass_wm_get_focused
telemetry.
- wm_menu_poll(539) and wm_popup_poll(541) now bypass BKL; their pending-id
handoff paths use atomic set/poll in WM to avoid races with compositor updates.
- wm_list_windows(546) now bypasses BKL, backed by wid-table locking in
wm::list_windows_wid() and dedicated syscall_bypass_wm_list_windows telemetry.
- Decomposition now also covers WM menu/popup extension syscall families
(wm_menu_set/wm_menu_poll/wm_popup_show/wm_popup_poll/wm_get_focused/wm_raise)
to isolate remaining desktop-control pressure from residual other.
- Decomposition also tracks wm_list_windows(546) and power control syscalls
(power_off(544), reboot(545)) to reduce residual other in desktop shell runs.

Project layout¶

TinyCoreOS/
├── iso/
│   └── limine.cfg              # Limine bootloader config (Multiboot2)
├── src/
│   ├── kernel/
│   │   ├── boot.asm            # 32→64-bit long mode, SSE, identity-map 0-1 GiB + 3-4 GiB
│   │   ├── linker.ld           # Kernel linked at 1 MiB, exports _kernel_end
│   │   ├── main.cpp            # kernel_main: full init sequence + idle loop
│   │   ├── console.{hpp,cpp}   # Dual-mode console: VGA text (fallback) + framebuffer
│   │   ├── panic.{hpp,cpp}     # kernel::panic + PANIC / ASSERT macros
│   │   ├── new.cpp             # operator new/delete -> heap::alloc/free
│   │   ├── multiboot2.h        # Multiboot2 tag structures (incl. framebuffer tag)
│   │   ├── io.hpp              # inb / outb port I/O
│   │   ├── gdt/                # 64-bit GDT + TSS descriptor, ltr
│   │   ├── idt/                # IDT, isr_stubs.asm, interrupt dispatch
│   │   ├── pic/                # 8259A PIC remap (IRQ0-15 -> vec 32-47)
│   │   ├── vfs/
│   │   │   ├── vfs.{hpp,cpp}   # VFS abstraction (Inode+uid, File, Superblock, FdTable, chmod/chown/rename)
│   │   │   └── ramfs.{hpp,cpp} # RamFS backend (heap-linked lists, setattr)
│   │   ├── fs/
│   │   │   ├── tinyfs.{hpp,cpp}# TinyFS v4: 4KiB-block FS, dynamic layout, file-level dedupe, double indirect, mtime, version
│   │   │   └── fat32.{hpp,cpp} # FAT32 driver (LFN, mtime, file version counter via nt_res)
│   │   ├── syscall/
│   │   │   ├── syscall.{hpp,cpp}      # MSR init, syscall_dispatch (SYS_EXIT..SYS_STAT)
│   │   │   └── syscall_entry.asm      # SYSCALL/SYSRET entry point + enter_userspace
│   │   ├── userspace/
│   │   │   └── user_init.asm          # Demo ring-3 program (position-independent)
│   │   ├── readme_data.asm            # Embeds README.md as .rodata blob → /tmp/readme.md at boot
│   │   ├── drivers/
│   │   │   ├── ata.{hpp,cpp}   # ATA PIO driver (28-bit LBA, primary bus, polling, 2 drives)
│   │   │   ├── fb.{hpp,cpp}    # Linear framebuffer driver (put_pixel, fill_rect, scroll_region_up)
│   │   │   ├── timer.{hpp,cpp} # PIT 100 Hz, ticks(), calls sched::tick
│   │   │   ├── keyboard.{hpp,cpp} # PS/2 scancode -> ASCII, Page Up/Down, arrow keys, Tab, F-keys
│   │   │   └── ac97.{hpp,cpp}  # Intel ICH AC97 audio: PCI scan, DMA ring, /dev/dsp, beep/play/volume
│   │   ├── gfx/
│   │   │   ├── font.{hpp,cpp}  # Embedded 8×8→8×16 bitmap font (ASCII 0x20-0x7E)
│   │   │   ├── canvas.{hpp,cpp}# Desktop chrome: title bar, live sysinfo, dynamic F-key bar (set_fkey_bar/reset_fkey_bar)
│   │   │   └── wm.{hpp,cpp}    # Window manager: compositor, drag/focus/resize, wid-based syscall API, dirty-flag rendering
│   │   ├── net/
│   │   │   ├── e1000.{hpp,cpp} # Intel 82540EM NIC driver (MMIO, TX/RX descriptor rings, polling)
│   │   │   ├── net.{hpp,cpp}   # Network stack: Ethernet II, ARP, IPv4, ICMP, UDP, DNS, TCP
│   │   │   ├── dns.{hpp,cpp}   # DNS A-record client (UDP port 53)
│   │   │   ├── httpd.{hpp,cpp} # HTTP/1.1 server (v2; static files, dir listing, runs as bg task)
│   │   │   ├── ftpd.{hpp,cpp}  # FTP server (plaintext, PASV, ACL + auth backends)
│   │   │   └── ssh.{hpp,cpp}   # SSH2 server + client (curve25519-sha256 + chacha20-poly1305 + ed25519)
│   │   ├── shell/
│   │   │   └── shell.{hpp,cpp} # Interactive kernel shell (70+ commands, history, tab completion,
│   │   │                       #   background jobs, startup scripts, users/groups, chmod/chown,
│   │   │                       #   aliases, stream redirection, pipes, NC file manager, dynamic F-key bar)
│   │   ├── zip/
│   │   │   └── zip.{hpp,cpp}   # ZIP reader/writer (Store + Deflate via miniz_tinfl)
│   │   ├── mm/
│   │   │   ├── pmm.{hpp,cpp}   # Physical Memory Manager (bitmap + alloc_n_frames)
│   │   │   ├── heap.{hpp,cpp}  # Kernel heap (first-fit free list, realloc, C malloc/free wrappers)
│   │   │   └── vmm.{hpp,cpp}   # Virtual Memory Manager (page-table walker, map/unmap/get_phys)
│   │   └── sched/
│   │       ├── task.hpp        # struct Task (RSP, privilege, FdTable, ring-list)
│   │       ├── scheduler.{hpp,cpp} # Round-robin, create_task/create_user_task, yield, kill_task, hint_idle
│   │       └── context.asm     # switch_context (save/restore 6 callee-saved regs)
│   ├── kstd/
│   │   ├── utility.hpp         # kstd::move, kstd::forward
│   │   ├── placement_new.hpp   # operator new(size_t, void*) for freestanding
│   │   ├── unique_ptr.hpp      # kstd::unique_ptr<T>
│   │   ├── vector.hpp          # kstd::Vector<T> - heap-backed dynamic array
│   │   └── cstring.hpp         # strlen, strcmp, strncmp, strncpy, strrchr, memcpy, memset, memmove
│   └── vendor/
│       ├── miniz/
│       │   ├── miniz_tinfl.{h,c}  # Deflate decompressor (richgel999/miniz)
│       │   ├── miniz_common.h
│       │   └── compat/         # Freestanding shim headers (stdlib, string, assert, …)
│       ├── stb/
│       │   ├── stb_truetype.h       # stb_truetype (nothings.org) — TTF rasterizer
│       │   ├── stb_truetype_impl.c  # Single-TU implementation (compiled as C)
│       │   └── compat/              # Freestanding shim headers
│       └── monocypher/
│           └── monocypher.{h,c}  # Monocypher 4.0.2 — X25519, ChaCha20, Poly1305, EdDSA (download: make monocypher-get)
├── userspace/
│   ├── libtwm.h                # Header-only user-space WM API (M19) - no libc required
│   └── filemanager.c           # Graphical file manager (M20) - musl ELF
├── tools/limine/               # Limine bootloader binaries (git clone, gitignored)
├── msys2-env.sh
├── Makefile
└── README.md

Architecture overview¶

Boot sequence¶

BIOS/UEFI
  └── Limine (Multiboot2)
        ├── reads iso/limine.cfg
        ├── loads kernel.elf into RAM
        ├── switches display → linear framebuffer (1024→768→32bpp VESA)
        └── jumps to boot.asm  [32-bit protected mode]
              ├── verify Multiboot2 magic (0x36D76289)
              ├── set up temporary GDT
              ├── enable PAE + paging
              -     ├── PDPT[0] → pd0: identity-map 0-1 GiB (2 MiB pages)
              -     └── PDPT[3] → pd3: identity-map 3-4 GiB (VESA FB BAR)
              ├── set EFER.LME → long mode (64-bit)
              ├── enable SSE (CR4.OSFXSR | CR4.OSXMMEXCPT)
              └── far-jump → kernel_main(mb2_addr)       src/kernel/main.cpp

Kernel init sequence (kernel_main - src/kernel/main.cpp)¶

Each step prints [OK] to the screen when it succeeds.

Shell startup (shell::run - src/kernel/shell/shell.cpp)¶

Runs as a separate scheduler task, waits for boot_done, then:

Timing (approximate, QEMU)¶

  0 ms   BIOS POST
500 ms   Limine loads kernel.elf from ISO
501 ms   boot.asm: long mode, paging
502 ms   kernel_main: console → PMM → heap → GDT → IDT → VMM → syscall
503 ms   VFS/RamFS → devfs → ATA → sched → PIC → PIT → keyboard
504 ms   WM → mouse → e1000 → tasks created → STI
519 ms   async USB stage-2 task starts (150 ms settle runs in background)
520 ms   shell::run() starts, boot_done received
521 ms   disk0 mount + disk layout bootstrap
522 ms   "login: " prompt  (or immediate shell if diskless)
670 ms   USB stage-2 block printed + login prompt repainted

Graphical Framebuffer (M8)¶

The Limine bootloader sets up a 1024 → 768 → 32 bpp VESA framebuffer.
QEMU uses -vga std (VBE 2.0). The framebuffer physical address
(typically 0xFD000000) is in the 3-4 GiB range, identity-mapped in
boot.asm via pd3 - no runtime page-table changes needed.

Desktop layout¶

-───────────────────────────────────────────────────────────────────
-  TinyCoreOS  M22 - Userspace Runtime Fixes   CPU 1%  RAM 210 MiB-  → 28 px title bar (live sysinfo top-right)
├──────────────────────────────────────────────────────────────────+
-                                                                  -
-  [OK] TinyFS v4: 4 KiB blocks, up to 2 TB partition, up to 4 GB file, file-level dedupe -
-  [OK] Disk 0 (TinyFS v4) mounted at /disk/disk0                 -
-  login: root                                                     -
-  CoreShell  (type 'help' or press F1 for README)                 -
-  root@tinycore:/$ _                                              -
-                      (terminal area - Page Up/Down scrolls)      -
├──────────────────────────────────────────────────────────────────+
- F1 Readme - F2 Files - F3 Tasks - F4 Net - → - F8 Monitor - F9 Cls-  → dynamic F-key bar
└──────────────────────────────────────────────────────────────────-

Font rendering¶

TinyCoreOS uses stb_truetype for antialiased
TrueType font rendering (see Font System for full details).

Default font: Hack Regular (embedded in the kernel binary as a C array;
always available regardless of filesystem state).

Glyphs are pre-rasterized at boot into an 8 → 16 px grayscale alpha cache
(95 printable ASCII chars). Each pixel is alpha-blended at draw time - no
per-character floating-point cost at runtime.

Box-drawing characters (U+2500 range, encoded as 0x80-0x8A internally) use the
original 1-bit bitmap renderer as a fallback since TTF fonts do not contain them.

Dual-mode console¶

CoreShell¶

Runs as a kernel task (ring 0). Features:

• Blinking cursor (_) via PIT tick counter
• Command history - ^/→ arrow keys; persisted to ~/.history per user
• Tab completion - cycles directories first, then files; prefix-aware
• Background jobs - append & to any command; jobs to list
• Console scrollback - Page Up / Page Down (200 rows)
• Dynamic F-key bar - context-sensitive function key menu at screen bottom

Dynamic F-key Bar (M21)¶

The bottom status bar shows function key labels that change with the active mode.
Pressing a labelled F-key either executes the command immediately or pre-fills
the command line so arguments can be added before pressing Enter.

Default mode

Files sub-menu (entered via F2; F10 returns to default)

Net sub-menu (entered via F4; F10 returns to default)

Logs sub-menu (entered via F5; F10 returns to default)

Custom sub-menu (entered via F8; F10 returns to default)

Slots F1..F9 are loaded from /disk/disk0/prefs/shell.prefs using:

• fkeys.custom.fN.label=<text>
• fkeys.custom.fN.action=inject:<cmd>|exec:<cmd>|builtin:none|builtin:clear

If a slot has no action, it remains visible but inactive.

Monitor sub-menu (optional / non-default mapping)

Filesystem commands¶

I/O operators¶

Pipe-aware built-ins (read from pipe when no file argument is given): grep, cat, tail.

Archive commands¶

Disk / filesystem commands¶

format size examples:

format 0          # first available drive (auto-resolve when vdrv0 is absent)
format 0 256M     # first 256 MiB only
format 0 4G       # first 4 GiB only
format 2 1T       # drive 2, first 1 TiB
format 0 512000   # raw sector count (512 000 → 512 B ? 256 MB)

Accepted size suffixes: K (KiB), M (MiB), G (GiB), T (TiB).
The rest of the physical disk beyond the specified size is left untouched.

TinyFS v4 dedupe smoke test:

format 0 /y
mount 0
write /disk/disk0/a "hello dedupe"
copy /disk/disk0/a /disk/disk0/b
tfs status /disk/disk0
ls -ac /disk/disk0

Expected:
- tfs status reports dedupe shared payloads >= 1 and non-zero dedupe saved bytes.
- Modifying one of the shared files triggers COW and increments dedupe COW events.
- After reboot + remount, tfs status still reports consistent dedupe counters.

Process management commands¶

System information commands¶

Network commands¶

User & security commands¶

Group commands¶

Permission commands¶

CoreShell GUI Apps¶

Two full-screen TUI applications are built directly into the kernel shell
(no busybox or ELF required). Each replaces the canvas F-key bar with its
own context while active, then restores the shell bar on exit.

nc - Not Commander¶

Dual-panel file manager inspired by Norton Commander.

nc

Layout

[menu bar] Left  File  Right  Commands  Options  Shortcuts
┌────────────── /left/path ─────────────┬───────────── /right/path ────────────┐
│ Name*  Size     T   Date     Ver      │ Name   Size      T   Date     Ver    │
│ ..                                  │ │ ..                                  │
│ Desktop/                            │ │ bin/                                │
│ Downloads/                          │ │ dev/                                │
│ ...                                 │ │ ...                                 │
└──────────────────────────────────────┴──────────────────────────────────────────┘
[status bar] panel stats + operation progress (copy/move/search/etc.)
[f-key bar] F1 Help  F2 Refresh  F3 View  F4 Edit  F5 Copy  F6 Move  F7 MkDir  F8 Del  F9 Menu  F10 Quit
[alt f-key bar] hold Alt: F5 ZIP  F6 UnZIP  F7 Find

Keyboard reference

F9 top menu

ftpc - FTP Commander¶

Norton-Commander-style dual-panel FTP client.
Left panel = local disk, Right panel = remote FTP server.

ftpc [session_name]

Sessions are saved to /disk/disk0/etc/ftpclient.conf (INI format)
and restored across invocations.

Keyboard reference

Session manager (F9)

Font System (M23)¶

Architecture¶

Kernel binary
  L→ hack_regular_data.h       → Hack Regular TTF embedded as uint8_t[]
       → font::init()
  src/vendor/stb/
  L→ stb_truetype_impl.c       → stb_truetype compiled as C (CFLAGS_STB, SSE2, -fno-builtin)
  L→ compat/                   → freestanding shims: string.h, stdlib.h, math.h, assert.h
       → stb_bake_glyph()
  src/kernel/gfx/font.cpp
  L→ s_cache[95][16][8]        → pre-baked grayscale alpha (12 KB, 95 ASCII glyphs)
       → font::draw_char()
  Alpha-blend over fb / window buffer

Glyph cache¶

At font::init() (called after heap::init()), stb_truetype rasterizes all 95
printable ASCII characters (0x20-0x7E) at 13 px height into an
s_cache[95][16][8] static array of 8-bit alpha values (? 12 KB total).

draw_char() / draw_char_to() then read from this cache with a fast integer
alpha-blend: out = fg → ?/255 + bg → (255-?)/255.

Box-drawing chars (0x80-0x8A) fall back to the original 1-bit bitmap renderer.

Runtime font switching¶

font                                        # list available fonts + show current
font /usr/share/fonts/JetBrainsMono.ttf    # switch immediately (re-bakes cache)

font::load_file(path) reads the TTF from VFS into a heap buffer, calls
stb_font_init(), re-bakes all 95 glyph cells, and frees the previous
heap-allocated font data. The change is instantaneous - the next
draw_char() call uses the new glyphs.

Adding fonts¶

Development / QEMU (manual):

mcopy -i build/fatdisk.img -o MyFont.ttf ::/usr/share/fonts/

Project fonts directory (fonts/):

cp MyFont.ttf fonts/
make fonts-to-fat        # requires mtools (pacman -S mingw-w64-x86_64-mtools)

Real hardware: include fonts in the disk image before flashing to USB/disk.

stb_truetype build notes¶

stb_truetype is compiled as a separate C translation unit with its own flags:

malloc/free in stb are redirected to heap::alloc/heap::free via
extern "C" wrappers already present in heap.cpp.

Networking¶

All networking is polling-based (no interrupt handler) on the Intel 82540EM
(e1000) emulated by QEMU (-device e1000).

Network config files¶

# /disk/disk0/etc/networks
[eth0]
mode    = dhcp

For static networking, switch to:

[eth0]
mode    = static
ip      = 10.0.2.15
netmask = 255.255.255.0
gateway = 10.0.2.2

# /disk/disk0/etc/resolv.conf
nameserver 10.0.2.3

HTTP Server (httpd)¶

A built-in HTTP/1.1 server (v2) that runs as a background scheduler task.

• Configuration: /disk/disk0/etc/httpd.conf (port, docroot, index file, mime mappings)
• Default document root: /disk/disk0/usr/www/
• Up to 6 concurrent HTTP sessions (tcp_srv pool); additional clients wait for a free slot instead of immediate forced disconnect
• Connection behavior is configurable in httpd.conf (max_sessions, idle_timeout_sec)
• Serves static files with correct MIME types (text/html, text/plain, image/*, →)
• Directory listing when no index file is present
• Path traversal protection (.. rejected with 403)
• Access log: /disk/disk0/log/httpd.log (timestamp, client IP:port, method/path, status, disconnect reason)
• QEMU forwards host port 18000 → guest port 80 (hostfwd=tcp::18000-:80) and 2222 → 22 for SSH

# /disk/disk0/etc/httpd.conf
port    = 80
docroot = /disk/disk0/usr/www
index   = index.html
max_sessions = 6
idle_timeout_sec = 5

# mime.types section
mime.html = text/html; charset=utf-8
mime.css  = text/css
mime.js   = application/javascript
mime.json = application/json
mime.png  = image/png
mime.jpg  = image/jpeg
mime.jpeg = image/jpeg

Usage:

netinit
httpd
# → open http://localhost:18000 in a host browser

FTP Server (ftpd)¶

Built-in FTP server running as a background scheduler task, with Explicit FTPS
(AUTH TLS / PBSZ 0 / PROT P) backed by mbedTLS.

• Configuration: /disk/disk0/etc/ftpd.conf
• Control channel can listen on multiple ports (port= repeated or ports=...)
• Passive mode (PASV) with configurable data-port range (pasv_port_start / pasv_port_end)
• Per-user roots and write permissions (user.<name>.root, user.<name>.write)
• Supports passwordless anonymous account
• IP filtering with allow/deny rules (allow_ip, deny_ip, wildcard * per octet)
• Directory/file operations include MKD, DELE and recursive RMD
• LIST exposes file type, permissions, size and modification timestamp (from VFS metadata)
• Optional authentication backend:
• auth_backend = local (passwords from ftpd.conf)
• auth_backend = shadow (verification against /etc/passwd + /etc/shadow)
• Built-in brute-force throttling:
• auth_fail_window_sec
• auth_max_failures
• auth_block_sec
• Explicit FTPS toggles in config:
• tls_enable = 0|1
• ftps_require_prot_p = 0|1
• FTPS runtime:
• TLS control-channel upgrade on AUTH TLS
• PROT P enables TLS on PASV data channel (LIST / RETR / STOR)
• Current certificate source: bundled mbedTLS test RSA certificate/key (self-signed)

Supported FTP commands

# /disk/disk0/etc/ftpd.conf
port = 21
#ports = 21,2121

max_sessions = 3
idle_timeout_sec = 60

pasv_port_start = 21000
pasv_port_end   = 21015

# IP ACL
#allow_ip = 127.0.0.1
#allow_ip = 10.0.2.*
#deny_ip  = 10.0.2.15

# Brute-force protection
auth_fail_window_sec = 300
auth_max_failures = 5
auth_block_sec = 180

# Auth backend: local | shadow
auth_backend = local

# Explicit FTPS policy (requires mbedTLS-enabled FTPS runtime)
tls_enable = 0
ftps_require_prot_p = 1

user.anonymous.root = /disk/disk0/usr/ftp
user.anonymous.password =
user.anonymous.write = 0

user.root.root = /disk/disk0/home/root
user.root.password =
user.root.write = 1

Usage:

netinit
ftpd start
ftpd status
ftpd stop

Startup Scripts¶

Two configuration files control automatic command execution:

/disk/disk0/etc/startup-services¶

Executed at boot before the login prompt, right after disk0 is mounted.
Each non-comment line is launched as a background task.

# /disk/disk0/etc/startup-services
# Uncomment to auto-start the web server at every boot:
# httpd
# ftpd
# netinit

/disk/disk0/etc/startup¶

Executed after each successful login as the logged-in user.
Each non-comment line is run as a foreground shell command.

# /disk/disk0/etc/startup
# cd /disk/disk0/home/root
# netinit

Edit both files with edit /disk/disk0/etc/startup.

Optional GUI integration: if /disk/disk0/prefs/startup.prefs exists, startup commands are filtered by its toggles (coredesk, network, httpd, ssh).
If the prefs file is missing, startup behavior stays exactly as before (shell-first path unchanged).

Dual-Disk Support¶

Both disks are blank images created on first make run
(build/tinyfs.img, build/tinyfs1.img).
The default QEMU size is configurable in the Makefile; TinyFS v4 supports
up to ~2 TB per partition. Use format 0 <size> to initialise only a
portion of a larger disk.

Disk layout on disk0¶

/disk/disk0/
├── home/                   → user home directories
│   └── root/
│       └── .history
├── etc/
│   ├── passwd              → user database       (user:uid:homedir)
│   ├── shadow              → password hashes     (user:8charsalt:16charhash)
│   ├── group               → group database      (groupname:gid:member1,member2)
│   ├── networks            → NIC configuration   (INI-style)
│   ├── resolv.conf         → DNS nameservers
│   ├── pci.ids             → custom PCI device DB (VVVV:DDDD=Vendor|Device)
│   ├── httpd.conf          → HTTP server config
│   ├── ftpd.conf           → FTP server config
│   ├── startup             → post-login commands
│   └── startup-services    → boot-time background services
├── log/
│   └── auth.log            → login / su audit trail
└── usr/
    ├── bin/                → user ELF programs (M10+)
    └── www/                → HTTP server document root
        └── index.html

Security & User Management¶

Authentication files¶

Password hashing¶

FNV-1a 64-bit mixed with an 8-character hex salt (from timer tick xorshift):

root:a1b2c3d4:e5f6a7b8c9d0e1f2
     ^^^^^^^^ ^^^^^^^^^^^^^^^^
     8-char   16-char hash of (password + salt)
     salt

Empty hash = no password. Old single-field entries (user:hash) accepted for
backward compatibility. Minimum password length: 4 characters.

Login security¶

• 3-attempt lockout: 3-second cooldown after 3 consecutive failures.
• Audit log: every login and su attempt written to auth.log.
• Root bypass: uid=0 can su to any account without a password.

File permissions¶

All inodes carry mode (rwxrwxrwx) and uid. Enforced in open, mkdir,
unlink:

• uid=0 (root) bypasses all checks.
• Owner: user bits (S_IRUSR/S_IWUSR).
• Others: other bits (S_IROTH/S_IWOTH).

TinyFS v4 persists mode, uid, ctime, mtime, version, and dedupe
metadata (content_crc32c, owner/refcount) to disk
(OnDiskInode). FAT32 stores ctime/mtime in standard directory-entry
date/time fields and repurposes nt_res for the version counter. Both RamFS and
TinyFS implement setattr for chmod/chown.

The version field in vfs::Inode / vfs::Stat is a write counter
(1 = freshly created, increments on each write(), saturates at 255).
It is displayed in nc (Ver column) and ls -a.

VFS¶

Filesystem tree¶

/                   → RamFS (always in RAM)
├── bin/
├── dev/
├── tmp/
└── disk/
    ├── disk0/      → TinyFS primary master (auto-mounted)
    └── disk1/      → TinyFS primary slave  (auto-mounted if formatted)

ATA PIO Driver¶

TinyFS v4 On-Disk Format¶

TinyFS v4 is a custom scalable filesystem with 4 KiB data blocks and a
fully dynamic metadata layout calculated at format time.

Disk layout (512-byte sectors)¶

Sector 0               Superblock  (magic "TFS4", version 4)
Sectors 1 .. B         Block bitmap  (1 bit per 4 KiB data block)
Sectors B+1 .. B+I     Inode bitmap
Sectors B+I+1 .. B+I+T Inode table  (4 inodes → 128 B per sector)
Sector  data_start+    Data blocks   (4 KiB = 8 sectors, block-aligned)

All offsets (bitmap_start, imap_start, itbl_start, data_start) are
stored in the superblock and computed at format time - no hardcoded positions.

Capacity¶

Key on-disk structures¶

struct OnDiskSb {           // 512 bytes, sector 0
    uint32_t magic;         // 0x54465334 ("TFS4")
    uint32_t version;       // 4
    uint32_t block_size;    // 4096
    uint32_t inode_count;   // total inodes
    uint64_t block_count;   // total 4 KiB data blocks
    uint64_t free_blocks;
    uint32_t free_inodes;
    uint32_t root_inode;    // always 0
    uint32_t bitmap_sectors, imap_sectors, itbl_sectors;
    uint64_t bitmap_start, imap_start, itbl_start, data_start;
    uint8_t  pad[424];
};

struct OnDiskInode {        // 128 bytes; 4 per 512-byte sector
    uint32_t mode, uid;
    uint64_t size;          // file size in bytes (uint64_t)
    uint32_t ctime, mtime;  // Unix timestamps
    uint32_t nlinks, blocks, flags;
    uint32_t direct[12];    // 4 KiB block numbers
    uint32_t indirect;      // single-indirect block
    uint32_t dindirect;     // double-indirect block
    uint32_t tindirect;     // triple-indirect (reserved)
    uint8_t  version;       // write counter (1-255)
    uint32_t content_crc32c;  // full-file hash for dedupe
    uint32_t dedup_owner_ino; // owner inode for shared payload
    uint32_t dedup_refcount;  // valid on owner inode
    uint8_t  dedup_flags;     // owner/shared flags
    uint32_t meta_crc32c;     // inode metadata checksum
    uint8_t  pad[14];
};

struct OnDiskDirent {       // 256 bytes; 16 per 4 KiB block
    uint32_t inode;         // 0 = unused
    uint16_t rec_len;       // always 256
    uint8_t  name_len;
    uint8_t  type;          // 1=file, 2=dir
    char     name[248];     // max 247 usable chars
};

Block addressing¶

Inode metadata fields¶

Every inode stores:
- ctime - creation time (Unix timestamp; filesystem-backed where available, fallback to mtime when unknown)
- mtime - last modification time (Unix timestamp, set by RTC on write)
- version - write counter, incremented on every write, saturates at 255, initialised to 1 at creation
- content_crc32c - full-file content hash used by TinyFS v4 file-level dedupe
- dedup_owner_ino / dedup_refcount - shared payload ownership and reference counting

vfs::stat() currently surfaces ctime, mtime, version, and metadata
checksum status. ls -a shows Created/Modified/Cks; nc and ls -a
show version.

How file-level dedupe works¶

TinyFS v4 deduplicates whole-file payloads (not variable chunks):

1. On write/truncate completion, filesystem computes content_crc32c.
2. Candidate files are filtered by same size + same content_crc32c.
3. Candidates are verified block-by-block (CRC match alone is never trusted).
4. On exact match:
   - current inode becomes shared member (dedup_owner_ino points to owner),
   - owner dedup_refcount is incremented,
   - duplicate payload blocks are released.
5. On future write to a shared file, copy-on-write (COW) detaches it first
   so modifications do not affect other files sharing the payload.
6. On unlink/truncate, refcount lifecycle is preserved:
   - non-owner unlink decrements owner refcount,
   - owner unlink elects replacement owner when needed,
   - physical blocks are freed only when effective refcount reaches zero.

You can force full re-deduplication with:

tfs dedupe scan /disk/disk0

And inspect live counters with:

tfs status /disk/disk0

Backward compatibility¶

TinyFS v4 uses a new magic/version (TFS4, version 4).
Older disks are recognised as "present but not formatted" and must be
reformatted with format [0|1] to use v4.

Text Editor (edit)¶

Full-screen terminal editor with line-oriented buffer and incremental redraw:

ZIP Support¶

miniz_tinfl.c compiled as a separate C translation unit with freestanding
shim headers in src/vendor/miniz/compat/.

Memory layout¶

Preemptive Scheduler¶

PIT fires IRQ0 at 100 Hz → sched::tick() → round-robin next Ready task →
switch_context.

switch_context(old_rsp*, new_rsp):
  push rbp/rbx/r12-r15   → save callee-saved regs on OLD stack
  [old_rsp*] = RSP        → store old RSP
  RSP = new_rsp           → load NEW stack
  pop r15-r12/rbx/rbp    → restore new task's regs
  ret                     → resume new task at saved RIP

CPU usage measurement¶

sched::tick() increments two counters on every PIT interrupt:

• s_total_ticks - always incremented
• s_busy_ticks - incremented only when the interrupted task was not idle

Tasks signal idleness by calling sched::hint_idle() immediately before a
hlt instruction or an unconditional sched::yield() with no work to do.
The next tick() sees the hint, skips s_busy_ticks, and clears the flag.
CPU% is sampled once per second: busy / total → 100.

Task lifecycle API¶

Syscall ABI (introduced in M6)¶

Historical Notes¶

Detailed milestone-by-milestone implementation notes (M10-M44) were moved out of this file to keep the main README focused on current usage.

• See docs/history.md for the archived summary.
• For exact low-level changes, use repository history (git log -- README.md and per-file history).

PHP Migration Plan¶

• Migration plan for bringing PHP into TinyCoreOS (MVP -> CGI -> hardening): docs/php_migration_plan.md

Make targets¶

Toolchain¶

Compile flags of note:
-march=x86-64 -mno-sse -mno-sse2 -mno-avx -mno-avx2 - kernel: baseline x86-64, no SSE (kernels must not use XMM registers)
-mno-red-zone - required for ring-0 interrupt handlers
-fno-stack-protector - no libgcc stack cookie in freestanding
-fno-threadsafe-statics - no __cxa_guard_* needed
-MMD -MP - automatic header dependency tracking
-fno-omit-frame-pointer - RBP chain preserved for kernel stack traces

stb_truetype (vendor C, separate flags):
-msse2 -fno-builtin - SSE2 enabled (font init runs before scheduler; XMM safe at boot);
-fno-builtin prevents GCC from silently emitting libm calls for sqrt/pow/cos

Network & TCP/IP (M30)¶

M30 completes the network stack with the missing components needed for automatic configuration and inter-process communication.

DHCP Client (net/dhcp.hpp/cpp)
- Full RFC 2131 / 2132 four-message exchange: DISCOVER → OFFER → REQUEST → ACK
- net::udp_send_bcast() added to net.cpp - sends from 0.0.0.0 to 255.255.255.255 via broadcast MAC before an IP is assigned
- On ACK: calls net::set_config(0, ...) with leased IP/mask/gateway and dns::set_server(0, ...) with the DNS server
- dhcp [seconds] shell command: manual lease request with live output and configurable timeout
- netinit automatically runs DHCP when /etc/networks has mode=dhcp
- VMware/NAT compatibility: client now accepts valid DHCP OFFER/ACK packets shorter than 548 bytes
- DHCP negotiation phases (DISCOVER/OFFER/REQUEST/ACK, timeouts, NAK) are mirrored to hdd.log for diagnostics

Loopback Interface (127.0.0.1)
- net::loopback_send() injects raw IP packets into an 8-slot ring buffer (no NIC, no Ethernet)
- net::poll() drains the loopback ring before the NIC RX ring
- handle_ip() now accepts broadcast (255.255.255.255) and loopback (127.x.x.x) destinations in addition to the interface IP
- ping 127.0.0.1 and localhost resolution work without network

/etc/hosts Lookup
- resolve_host() wrapper in shell.cpp checks /disk/disk0/etc/hosts before calling dns::resolve()
- Standard format: <ip> <hostname> [<alias>...]; comments with #
- All commands that resolve hostnames (ping, wget, ftp, ftpc, smbclient, resolve) use the wrapper
- localhost resolves to 127.0.0.1 without a file (hardwired in dns::resolve)

AF_UNIX Domain Sockets (net/socket.hpp/cpp)
- AF_UNIX = 1, struct SockAddrUn { sun_family; sun_path[108] }
- KSock extended with domain, unix_pair, unix_side, unix_path
- Socket pool enlarged from 4 to 16 slots; 8 UnixPair ring-buffer pairs (8 KiB each direction)
- unix_bind(path) - registers path in binding table, creates VFS placeholder inode
- unix_connect(path) - finds listener, allocates pair, connects side A
- unix_accept(listener) - creates new KSock for side B, returns File*
- unix_read/unix_write - bidirectional ring buffers (A→B and B→A)
- Syscall layer updated: SYS_BIND, SYS_CONNECT, SYS_ACCEPT/ACCEPT4 dispatch on KSock::domain

Shell I/O Plumbing (M31)¶

M31 adds the syscalls required for a user-space shell to set up I/O redirection and pipes, as well as standard file-manipulation operations that musl/libc programs rely on.

dup / dup2 / dup3 (syscalls 32 / 33 / 292)
- vfs::File gains a refcount field (starts at 1); vfs::close() only destroys when it reaches 0
- FdTable::replace(new_fd, f) - atomically installs f and returns the old occupant for closing
- fork() and clone() bump the refcount for every inherited fd
- dup2(old_fd, new_fd) closes the previous file at new_fd before installing the new reference
- dup3 adds O_CLOEXEC semantics; requires old_fd != new_fd
- SYS_READ/SYS_WRITE now check fd_table[0/1/2] first - dup2 redirections to stdin/stdout work correctly

pread64 / pwrite64 (syscalls 17 / 18)
- Read/write at an arbitrary offset without changing f->pos
- Saves and restores f->pos around the underlying VFS call
- Required by sqlite3, busybox's dd, and many database-like programs

truncate / ftruncate (syscalls 76 / 77)
- vfs::FileOps gains a truncate(File*, int64_t new_size) callback (7th field)
- ramfs: file_truncate() grows (zero-fills) or shrinks the buffer; adjusts f->pos if beyond new size
- tinyfs / fat32: callback is nullptr; VFS generic fallback handles shrink-only via inode->size
- vfs::truncate(path, size) opens the file internally and delegates to ftruncate

rename / renameat / renameat2 (syscalls 82 / 264 / 316)
- vfs::rename() was already implemented in vfs.cpp; this adds the syscall dispatch
- renameat/renameat2 resolve paths relative to dirfd (AT_FDCWD treated as cwd)
- Required by editors, package managers, and any program using atomic save-to-temp patterns

getpgrp (syscall 111)
- Returns sched::current()->pgid; needed for job-control queries by shells

FD_MAX increased from 16 → 64
- Busybox and most programs need more than 16 simultaneous open files
- FdTable size increases from 128 to 512 bytes per task (heap-allocated, no stack impact)

PTY - Pseudo-Terminal (M32)¶

M32 adds a complete pseudo-terminal subsystem, enabling user-space shells (busybox sh, bash) to run interactively with job control, line editing, and proper terminal semantics.

Architecture (src/kernel/tty/pty.hpp / pty.cpp)
- Up to 4 simultaneous PTY pairs; each pair has two 4 KiB ring buffers:
- m2s (master→slave): keyboard/terminal-emulator input going to the program
- s2m (slave→master): program output going back to the terminal
- Per-pair tty::Termios struct - each PTY has its own independent terminal settings

/dev/ptmx - PTY master
- open("/dev/ptmx") / posix_openpt() is intercepted in SYS_OPEN/SYS_OPENAT before VFS
- Each open allocates a fresh slot and returns a master File* with custom FileOps
- ioctl(master_fd, TIOCGPTN, &n) → returns the PTY index n
- ioctl(master_fd, TIOCSPTLCK, &0) → unlocks the slave so it can be opened
- TCGETS/TCSETS on the master operate on the slave's termios

/dev/pts/N - PTY slave
- open("/dev/pts/N") is intercepted; returns nullptr if slot n is locked or unused
- ioctl(slave_fd, TIOCSCTTY, 0) - sets controlling terminal (stub; updates session)
- ioctl(slave_fd, TCGETS/TCSETS) - reads/writes the per-PTY termios
- TIOCGWINSZ / TIOCGPGRP delegate to global console state

N_TTY line discipline (slave side)
- Canonical mode (ICANON): slave_read blocks until \n or ^D; slave_poll_read scans the ring for a complete line
- Raw mode: bytes are delivered immediately on each vfs::read
- Echo (ECHO/ECHOE): input written to m2s is echoed back to s2m for the master to display; DEL (0x7F) sends BS SP BS erase sequence
- ICRNL: CR → NL on master write (input processing)
- ONLCR: NL → CR+NL on slave write (output post-processing)

Blocking reads
- Both master and slave read ops call sched::yield() until data is available - safe because SYS_READ enables interrupts (STI) before calling vfs::read

Busybox on Disk (M33)¶

M33 delivers a complete user-space toolchain on disk via the statically-linked busybox multi-call binary, plus the kernel infrastructure needed to run it correctly.

ELF loader - full Linux ABI stack (src/kernel/exec/elf.cpp)

elf::load now accepts argv[], argc, envp[], envc. The initial user-stack layout strictly follows the System V AMD64 ABI:

RSP → [argc]
      [argv[0] ptr] → [argv[argc-1] ptr] [NULL]
      [envp[0] ptr] → [envp[envc-1] ptr] [NULL]
      [AT_PAGESZ, 4096] [AT_HWCAP, 0] [AT_CLKTCK, 100]
      [AT_HWCAP2, 0] [AT_SECURE, 0] [AT_RANDOM, ptr] [AT_NULL, 0]
[high VA]  argv/envp strings (packed) + 16-byte random block

A built-in minimal environment is injected when envp is not provided:
HOME, PATH=/bin:/usr/bin:/disk/disk0/bin:/disk/disk0/usr/bin, TERM=vt100, SHELL=/bin/sh, USER=root.

SYS_EXECVE - user-space argv/envp (src/kernel/syscall/syscall.cpp)

The execve handler now copies the user-space char **argv and char **envp arrays into kernel buffers (heap-allocated, 8 KiB each, up to 64 strings) and passes them to elf::load. On failure the buffers are freed before calling sched::exit_current.

Shell - cmd_exec argv splitting

cmd_exec(arg, argv0_override) splits the tail of the exec argument string into tokens and builds a proper const char* argv[]. The optional argv0_override allows the PATH dispatcher to pass the typed command name (e.g. sh) as argv[0] even when the resolved binary is /disk/disk0/bin/busybox - this is the mechanism that makes busybox applet dispatch work via symlinks.

Shell - PATH search extended

The implicit PATH search now covers four directories in order:
1. /bin
2. /usr/bin
3. /disk/disk0/bin
4. /disk/disk0/usr/bin

bb-install shell command

bb-install [url]

1. Downloads busybox via TFTP from 10.0.2.2:69 (QEMU built-in, no extra server needed) - or via HTTP if a full URL is passed
2. chmod 755 /disk/disk0/bin/busybox
3. Shows an interactive group-selection menu (Space = toggle, Enter = install, Esc = cancel):

nc (netcat) is always excluded - the name conflicts with the built-in Not Commander (nc).

Interactive shell fix (M33 post) - fully working ?

After the initial M33 implementation busybox sh loaded but exited immediately, and later ran but showed no prompt. The root causes were a sequence of missing kernel behaviours fixed iteratively:

Build guard - make run requires busybox

make run and make serial now call check-busybox first. If build/busybox is missing, make stops with a clear error and instructions. Python is no longer required for any build or run step (only optionally for make serve).

Complete M33 workflow

# Step 1 (host, once): get the busybox binary into build/
make busybox-prebuilt   # MSYS2/Windows: download pre-built x86-64 binary
# OR: make busybox      # WSL2/Linux: compile with musl-gcc
# OR: make busybox-docker

# Step 2 (host): start QEMU - TFTP is built in, no Python/HTTP server needed
make run

# Step 3 (kernel shell, first boot)
format 0            # initialise TinyFS on disk0 (once)
mount 0             # mount disk0 at /disk/disk0
netinit             # bring up eth0 (DHCP via QEMU SLIRP)
bb-install          # TFTP busybox + chmod 755 + interactive group-selection menu for symlinks

# Step 4: launch interactive busybox shell
sh                  # → busybox sh prompt  / #
ls /                # list root filesystem
echo hello world    # basic test

TFTP vs HTTP: bb-install uses QEMU's built-in TFTP server (10.0.2.2:69) which serves the build/ directory automatically - no make busybox-serve / Python needed. Pass an HTTP URL to use wget instead: bb-install http://10.0.2.2:9090/busybox.

CoreShell CLI Completeness (M35)¶

M35 rounds out the CoreShell built-in command set so day-to-day shell usage does not require busybox.

New built-in commands

Alias expansion

All aliases are expanded before dispatch, so user-defined shortcuts work transparently for every built-in and external command.

lsproc enhancements

lsproc now displays per-process memory usage and a full status legend:

PID  NAME             STATE PRIV  STK_KB  MEM_KB  CWD
──────────────────────────────────────────────────────
  1  init             S     K        64      64   /
  2  shell            R     K       128     132   /disk/disk0/home/root
  3  compositor       S     K        64      64   /

Legend:
  STATE: R=Running  S=Sleeping  D=Disk-wait  Z=Zombie
  PRIV:  K=Kernel   U=User      *=own address space
  MEM_KB = kernel stack + task_brk heap (user tasks only)

SSH Client & Server (M36)¶

M36 adds a full SSH2 implementation, enabling encrypted remote access to TinyCoreOS from any OpenSSH-compatible client.

Architecture (src/kernel/net/ssh.hpp / ssh.cpp)¶

The SSH implementation lives entirely in the kernel (ring-0). The server (sshd) runs on the TCP server pool (tcp_srv_listen/tcp_srv_accept) with a small worker pool (up to 5 concurrent sessions), while client mode uses the active TCP path, allowing stable local self-connect tests (ssh localhost) and concurrent service operation.

Recent reliability/UX updates:
- Per-session SSH scratch buffers replaced shared static packet buffers to avoid client/server data races.
- KEX flow was adjusted so the client sends KEXINIT immediately after version exchange (prevents localhost deadlock).
- SSH banner identity is built from release macros (TCOS_PRODUCT_NAME, TCOS_PRODUCT_CODENAME, TCOS_PRODUCT_VERSION) with safe token sanitization.
- Interactive shell output is buffered/chunked (instead of per-byte sends) for better stability.
- SSH shell prompt is visibly different: (ssh) $.

Cryptography stack

SSH protocol subset

Host key persistence

On first sshd launch the server generates a random Ed25519 key pair and saves the private key to /disk/disk0/etc/ssh/ssh_host_ed25519_key. Subsequent starts load the saved key so the host fingerprint remains stable across reboots.

Shell integration

SSH sessions execute commands via shell::exec_cmd() (exported from shell.cpp). Output is captured with console::set_char_hook, normalized to CRLF for terminal clients, and sent in buffered chunks.

Commands¶

sshd start                            # start SSH server on port 22 (background task)
sshd status                           # check state
sshd stop                             # stop service
sshd restart                          # restart service
ssh [user@]host [command]             # connect to remote SSH server; masked password prompt (`*`)

Usage example¶

# Host (QEMU port-forwarded to 2222):
ssh -p 2222 root@localhost
# → prompts for password
# → interactive CoreShell session over encrypted SSH

# Self-connect inside TinyCoreOS for local debugging:
ssh localhost

# Run a single command:
ssh -p 2222 root@localhost "lsproc"

Monocypher vendor library¶

Monocypher is downloaded once via:

make monocypher-get    # fetches monocypher.c + monocypher.h from GitHub (tag 4.0.2)

The library is compiled as a plain C translation unit with -std=c11 -Os -fno-builtin and linked into the kernel. It is checked as a build prerequisite → a missing monocypher.c produces a clear error with instructions.

AC97 Audio (M37)¶

M37 adds a complete AC97 audio subsystem, enabling sound output from the kernel and user-space programs.

Architecture (src/kernel/drivers/ac97.hpp / ac97.cpp)¶

The driver targets QEMU's emulated Intel ICH AC97 codec (-device AC97, PCI 0x8086:0x2415).

Hardware layout

DMA ring buffer

BDL: 32 → 8-byte entries
     [phys_addr : 32] [count : 16 (→ int16_t)] [flags : 16]

DMA buffers: 32 → 4096 bytes = 128 KiB contiguous (pmm::alloc_n_frames)
Audio format: 48000 Hz, 16-bit signed, stereo (4 bytes / frame)
Buffer duration: 4096 / 4 / 48000 ? 21 ms per buffer
Ring capacity:  32 → 21 ms ? 680 ms

Playback flow

1. Copy PCM data into DMA buffers (up to 32 per batch)
2. Write BDL physical address to NABM_PO_BDBAR
3. Set NABM_PO_LVI = last filled buffer index
4. Set NABM_PO_CR bit 0 (RPBM) → DMA starts
5. Poll NABM_PO_SR bit 0 (DCH = halted) or yield via sched::yield()
6. Repeat for remaining data

Variable Rate Audio (VRA)

If the codec reports VRA support (NAM Extended Audio ID bit 0), the driver enables it and programs the front DAC sample rate register to 48000 Hz.

/dev/dsp

Registered as a VFS inode with custom FileOps. Writing raw PCM bytes calls ac97::play_pcm() directly → compatible with simple OSS-style audio programs.

Shell commands¶

WAV support: the play command detects RIFF/WAV headers, validates 16-bit stereo format, and skips the standard 44-byte header automatically.

QEMU setup¶

# Makefile already adds -device AC97 to QEMU_FLAGS
make run

# Inside the OS:
beep               # 880 Hz beep (tests audio output)
beep 440 1000      # A4 note for 1 second
volume 80          # reduce volume to 80%
play /disk/disk0/sounds/test.wav

Roadmap¶

Phase 1 -> Widget Toolkit & Screen System (M39->M40) ✅¶

Foundation of the entire GUI stack. Without this layer, no graphical component can exist.

Phase 2 -> Workbench Core (M41->M43) ¶

Desktop with icons, drawers, and top-screen menus: the heart of the AmigaOS-style experience.

Phase 3 -> System Applications (M44->M47)¶

Daily workflow tools: shell, preferences, multimedia viewer, and dock.

Phase 4 -> Polish & Tooling (M48->M50)¶

Visual polish, text editor, and system notifications.