Make sure your notebooks are synced to the cloud. If you lose your physical USB drive, your data will still be completely safe.
Microsoft’s business model relies on user accounts, telemetry, and cloud subscriptions. A truly portable app that leaves no trace would bypass Windows licensing checks, OneDrive upsells, and usage analytics. It is commercially counterintuitive for them. Microsoft Onenote Portable