init: auto-size dbcache to available system RAM

[kwsantiago]

Auto-size the default dbcache to half of system RAM (capped at 16 GiB) instead of the hardcoded 450 MiB default. Falls back to 450 MiB on platforms where RAM detection fails.

Also enables -lowmem by default with a 256 MiB threshold so caches are flushed before the system runs out of memory.

Platform support: Linux (sysinfo), Windows (GlobalMemoryStatusEx), macOS (sysctlbyname).

Closes #70

[luke-jr]

How does this compare to bitcoin#34641 ?

[kwsantiago]

@luke-jr Same idea, although this PR is more aggressive. This PR uses min(RAM/2, 16G) vs their clamp(100M, 0.25*(RAM-2G), 2G), paired with -lowmem on by default as a safety net.

[pdath]

My first thought is - who are we optimising the default dbcache for? Examples include:

• home users running it on their Windows computer (along with lots of other apps)
• home miners running it on low-power, low-ram machines, along with some kind of "pool" software
• more serious users running it on a dedicated machine
• professional miners running it on high RAM servers

IMHO, home users should not be expected to know how to configure Bitcoin Knots' dbcache. I believe the configuration should be made conservative and safe for this group. Basically, we are making choices on their behalf.
IMHO, serious and professional users should know how to configure Bitcoin Knots' dbcache.

For these reasons alone, autosizing to use half of the system's physical RAM is a "Concept NAK" for me.
I have no strong feelings about the "clamp(100M, 0.25*(RAM-2G), 2G)" option.

I've found lookups via dbcache are O(1). Lookups via LevelDB are O(log(n)).
There is definitely a mathematical advantage to using a larger dbcache for the "hot" transactions.
However, as you increase dbcache, you take RAM away from the OS for general file caching. It is not as simple as "bigger dbcache is faster". It can, in fact, hurt the performance because it starves the OS file system cache.

My gut feeling is, for home users (who are the only group we are making this decision for), that no more than 25% of system RAM should be auto-allocated for Bitcoin Knots.