Packaging cleanups ("syslibs")

Adds options to build using system LevelDB and libsecp256k1 libraries (see ./configure --help for usage).

Please note this may be dangerous and make theft from you possible if used with versions of the LevelDB or libsecp256k1 libraries which have not been audited to meet the bug-for-bug compatibility required by Bitcoin's consensus protocol. These options are disabled by default for this reason.

It also fixes minor bugs that affect building only.

See also:

• Support for building against system LevelDB
• configure: Add unsupported --with-system-libsecp256k1 configure flag
• configure: BITCOIN_SUBDIR_TO_INCLUDE: Improve compatibility with paths including space and multiline cpp output
• crypto/sha256: Use pragmas to enforce necessary intrinsics for GCC and Clang
• test: Support -cli tests using external bitcoin-cli
• build: ignore deprecated-declarations warnings in objc++ macOS code
• upnp: fix build with miniupnpc 2.2.8
• Bugfix: Move common/url to libbitcoin_util.a for now

Fixes

• rpc: Don't FlushStateToDisk when pruneblockchain(0)
• Escape rather than remove any printable characters in UAs
• Try to use posix_fadvise with CBufferedFile
• http: Fail initialization when any bind fails
• Bugfix: GUI: Remove broken ability to edit the address field in the sending address book
• Bugfix? Restore linking to libmingwthrd
• Bugfix: devtools/symbol-check: Check PE libraries case-insensitively
• Fix release tarball generated by gitian
• Bugfix: Only use git for build info if the repository is actually the right one
• wallet: let Listwalletdir do not iterate through our blocksdata
• util/system: Close non-std fds when execing slave processes
• Bugfix: Skip tests for tools not being built
• Bugfix: util: Correctly handle Number value types in GetArg/GetBoolArg
• rpc: getblock/getrawtransaction/decode*/gettxout fixups
• blockstorage: do not flush block to disk if it is already there
• Blocking arguments -nohelp, -noh, and -no?
• CLI: Only one Request Handler can be specified
• exclude ipc scheme from port check
• Fix issues in ZMQ error handling
• Bugfix: net_processing: Restore "Already requested" error for FetchBlock
• security: restrict abis in bitcoind.service
• Bugfix: Package relay / bytespersigop checks
• Bugfix: configure: Correct check for fuzz binary needing a main function
• doc: fixup help output for -upnp and -natpmp
• wallet, rpc: add anti-fee-sniping to `send` and `sendall`
• Bugfix: RPC: Check for blank rpcauth on a per-param basis
• rpc: validate fee estimation mode case insensitive
• util: explicitly close all AutoFiles that have been written
• Drop log category in `SeedStartup`
• cli: Detect port errors in rpcconnect and rpcport
• wallet: default wallet migration, modify inconvenient backup filename
• BufferedFile: fclose at destruction
• Bugfix: GUI: Help messages already have a trailing newline, so don't add an extra one
• Bugfix: Correct first-run free space checks
• depends: build miniupnpc with CMake
• assumeutxo: Fix -reindex before snapshot was validated
• net: Decrease nMaxIPs when learning from DNS seeds
• psbt: Check non witness utxo outpoint early
• JSON-RPC request Content-Type is application/json
• chainparams: Add achow101 DNS seeder
• depends: qt 5.15.14 and fix macOS build with Clang 18
• net: Allow -proxy=[::1] on nodes with IPV6 lo only
• wallet: use LogTrace for walletdb log messages at trace level
• Fix cases of calls to `FillPSBT` errantly returning `complete=true`
• net: fix race condition in self-connect detection
• init: change shutdown order of load block thread and scheduler
• fix: Make TxidFromString() respect string_view length
• rest: Reject negative outpoint index early in getutxos parsing
• doc: getaddressinfo[isscript] is optional
• rest: Reject truncated hex txid early in getutxos parsing
• log: Remove NOLINT(bitcoin-unterminated-logprintf)
• ci: add `_LIBCPP_REMOVE_TRANSITIVE_INCLUDES` to TSAN (libc++) job
• guix: move bison from global scope, to Linux
• test: fix constructor of msg_tx
• Initialise DBus notifications in another thread
• Bugfix: Allow the user to start anyway when loading a wallet errors
• QRImageWidget: Sizing and font fixups
• Avoid unclean exit due to permissions issues when setting start on system startup
• Translate unit names & fix external signer error
• Fix shortcut ambiguities
• Fix transaction view/table
• Bugfix: Address broken things around Peers detail view
• Improve 'Requested Payments History' Multiselect
• debugwindow: update session ID tooltip
• Keep focus on "Hide" while ModalOverlay is visible
• Bugfix on TransactionsView - Disable if privacy mode is set during wallet selection
• Fix misleading signmessage error with segwit
• OptionsDialog: Allow Maximize of window
• OptionsDialog: Prefer to stretch actual options area rather than waste space
• Rendering an amp characters in the wallet name for QMenu
• RPC: Support specifying different types for param aliases
• Wallet: Hacky fix for getbalance bugs
• Bugfix: GUI/Addressbook: Drop confusing message about signed messages
• RPC/Net: Allow addconnection on non-regtest networks
• Add workaround for Boost bugs
• Re-enable external signer support for Windows
• Allow acceptstalefeeestimates on all networks
• zmq: accept unix domain socket address for notifier
• GUI: Avoid unnecessary BIP 21 references (Knots supports BIP 20 also)
• RPC: Keep .cookie if it was replaced after being generated
• RPC/Blockchain: Demote loadtxoutset to hidden/debug-only for now (not supported on mainnet)
• Bugfix: RPC: Return all warnings in get{blockchain,mining,network}info
• p2p, bugfix: correctly detect CJDNS addnode entries in AddNode()
• Sanity check to detect LLVM bug 96267

Functional and policy enhancements

• Parse URIs with non-BTC amounts
• -acceptnonstdtxn option to skip "non-standard transaction" checks
• Support for Tonal Bitcoin units (ᵇTBC, ˢTBC, and TBC)
• Restore ability to display addresses in GUI
• [Qt] support for persisted rpc console history
• Qt: Add network port input box to GUI settings
• Make RBF policies optional
• Render icons from SVG
• Read/write bitcoin_rw.conf for exposing shared Daemon/GUI options in the GUI
• RPC: sendrawtransaction: Allow the user to ignore/override specific rejections
• Add mempool statistics collector
• [Qt] Add interactive mempool graph
• RPC: Add parameter to addmultisigaddress / createmultisig to sort public keys
• Wallet/RPC: sweepprivkeys method to scan UTXO set and send to local wallet
• Drop IO priority to idle while reading blocks for peer requests and startup verification
• [RPC] dumpmasterprivkey command
• If -spkreuse=0, ensure transactions in mempool always have unique scriptPubKeys
• Expire bitcoind & bitcoin-qt 7-8 years after its last change
• Added support for MSG_FILTERED_WITNESS_BLOCK messages
• ZMQ: add publishers for wallet transactions
• Relax punishment for peers relaying invalid blocks and headers
• RPC: Allow rpcauth configs to specify a 4th parameter naming a specific wallet
• Add new bitcoin_rw.conf file that is used for settings modified by this software itself
• Multiselect in coincontrol treewidget and display selected count
• [REST] add a rest endpoint for estimatesmartfee, docs, and test
• Bugfix: RPC/Wallet: Include HD key metadata in dumpwallet
• Wallet: Support disabling implicit Segwit operation
• Add RPC call setscriptthreadsenabled: allow to temp. throttle CPU usage
• Add POWER8 ASM for 4-way SHA256
• gui: Add Windows taskbar progress
• rpc: Add min_conf option to fund transaction calls
• validation: sync chainstate to disk after syncing to tip
• torcontrol: Launch a private Tor instance when not already running
• GUI: Add Pairing tab with Tor onion address as copyable text and QR code
• Add feerate histogram to getmempoolinfo
• Restore warning for individual unknown version bits, as well as unknown version schemas
• qt: Add -guisettingsdir option
• gui: Shortcut to close RPC Console
• rpc: query general daemon information via RPC
• lib: Optimizing siphash implementation
• Add new filter type v0 for segwit only Scripts to blockfilterindex
• RPC: Show fee in results for signrawtransaction* for segwit inputs
• net: Add blockfilters white{bind,list} permission flag
• cli, test, doc: bitcoin-cli -getinfo multiwallet balances follow-ups
• cli: display multiwallet total balance in -getinfo
• RPC: Add getrpcwhitelist method
• Add -uaappend option to append a literal string to user agent
• Prune locks
• Flush dbcache early if system is under memory pressure
• wallet: introduce setfeerate (an improved settxfee, in sat/vB)
• rpc: Support -rpcauthfile argument
• RPC/Net: Allow changing the connection_type for addnode onetry
• rpc: Add getblocklocations call
• wallet: indicate whether a transaction is in the mempool
• - Add maxmempool RPC
• wallet: allow toggling external_signer flag
• rpc: add period_start to version bits statistics
• Add reindex=auto flag to automatically reindex corrupt data
• build: Add --with-append-cxxflags option
• Support multiple -*notify commands
• build: Use guix within gitian
• RPC/Wallet: Add "use_txids" to output of getaddressinfo
• util: Add "importfromcoldcard" command to bitcoin-wallet tool
• add support to save fee estimates without having to shut down the node
• rpc: add require_checksum flag to deriveaddresses
• rpc: allow dumptxoutset to dump human-readable data
• RPC/Wallet: Convert walletprocesspsbt to use options parameter
• rpc: Filter inputs by type during CoinSelection
• rpc,rest,zmq: faster getblock, NotifyBlock and rest_block by reading raw block
• rpc, p2p: allow `disconnectnode` with subnet
• cli: improve error message on multiwallet and add validation to cli-side commands
• rpc: make importaddress compatible with descriptors wallet
• p2p: Allow whitelisting manual connections
• wallet: Add wallet method to detect if a key is "active"
• wallet: add `seeds` argument to `importdescriptors`
• Allow configuring target block time for a signet
• net: Add new permission `forceinbound` to evict a random unprotected connection if all slots are otherwise full
• Introduce 'getblockfileinfo' RPC command
• init: Add option for rpccookie permissions (replace 26088)
• validation: don't clear cache on periodic flush: >2x block connection speed
• datacarriersize: Match more datacarrying
• fix: unnecessary continuation after finding mutation
• p2p: Increase tx relay rate
• wallet, rpc: document and update `sendall` behavior around unconfirmed inputs
• RPC: add new `listmempooltransactions`
• policy: enable sibling eviction for v3 transactions
• Add a `-permitbarepubkey` option
• rpc/net: Adds misbehaving_score to getpeerinfo
• contrib: list other binaries in manpage output
• refactor: Reserve memory for ToLower/ToUpper conversions
• Bugfix: bitcoin-cli: Check length of peer.transport_protocol_type
• Update manpage descriptions
• cli: improve bitcoin-cli error when not connected
• guix: build GCC with --enable-standard-branch-protection
• policy: restrict all TRUC (v3) transactions to 10kvB
• dbwrapper: Bump LevelDB max file size to 128 MiB to avoid system slowdown from high disk cache flush rate
• Add option dbfilesize to control LevelDB target ("max") file size
• net: add ASMap info in `getrawaddrman` RPC
• rpc: avoid copying into UniValue
• refactor: performance-for-range-copy in psbt.h
• rest: don't copy data when sending binary response
• optimization: Moved repeated `-printpriority` fetching out of AddToBlock
• rpc: add utxo's blockhash and number of confirmations to scantxoutset output
• Intro: Have user choose assumevalid
• Support backup to new text-based database dump format
• Network Graph layout - debug window improvement
• Make row color alternating in the Peers tab optional
• Peers window: Show direction in a new column, with clearer icon
• Network Watch tool
• Enable a non-linear network traffic option (click to toggle between linear and non-linear)
• Show ToolTip on Network Traffic graph
• Point out position of invalid characters in Bech32 addresses
• Warn when sending to already-used Bitcoin addresses
• Showing Local Addresses in Node Window
• Enable customisation of QR Code font
• Show maximum mempool size in information window
• add `-rejectparasites` option
• Remove embedded libminisketch (not used at all in 27.x)
• txdb: Increase max dbcache size to 1 TB
• GUI: Alt-P shortcut key for paste button in Open URI dialog
• RPC/rawtransaction: Convert descriptorprocesspsbt to use options parameter
• utilioprio: Add Windows support as ioprio_set_file_idle
• crypto/sha256: Use pragmas to enforce necessary intrinsics for GCC and Clang (POWER8)
• RPC: Backward compatibility with Knots 0.19-0.21.0 validateaddress (address_type param and error_index output)
• RPC: getrpcwhitelist: Return a list of wallets as a JSON Object
• wallettool: Warn about dump commands not fully dumping/restoring BDB wallets
• Revert "Remove -feefilter option"
• GUI: Convert payment request to show in QTextEdit
• RPC: add transaction hash to mempool entry output
• wallet: Support %w in -walletnotify script, on Windows
• Restore blockmaxsize option, allowing to limit mined blocks by byte size
• Policy: Restore support for mining based on coin-age priority
• Add mempooltruc=reject/accept/enforce option to enable TRUC support
• GUI/Wallet: Strip ".dat" from end of wallet display names
• GUI: Restore "request payment" button text
• GUI: Peers: When sorting by address, sort by network first
• GUI/Peers: Drop redundant Network column
• RPC/blockchain: Restore ability for pruned nodes to getblockfrompeer future blocks
• RPC/blockchain: Allow using getblockfrompeer without already having the block header
• RPC/Mining: Support overriding blockmaxsize, blockmaxweight, minfeerate options, and skip_validity_test in RPC params
• Revert "libconsensus: deprecate"
• Build DMG disk image for macOS targets
• Restore original bytespersigop as bytespersigopstrict
• Add dustdynamic option supporting fee estimator or kvB into mempool
• Add -datacarriercost option to adjust policy vsize of datacarrier bytes
• Policy: Add acceptnonstddatacarrier option to reject non-standard datacarrier regardless of size
• Add rejecttokens policy option to filter out Runes
• Add maxscriptsize policy option
• Wallet: Remove deprecation warning for newly created legacy wallets
• wallet: Preserve unsupported key origin flags
• net_permissions: Include "addr" permission on implicit whitelist entries
• RPC/blockchain: Add getblockfrompeer nodeid alias for backward compatibility
• GUI: Make Peers table column/splitter size keys uniquely named for Knots 23.0
• Wallet: Increase default confirmation target to 144 blocks (1 day)
• Revert "Disable bloom filtering by default."
• Wallet: Avoid using change types newer than user's preferred address type
• Enforce checkpoints at their specific block height
• Update checkpoints and chain params, adding a new checkpoint at block 855,000
• Make most policy options configurable in GUI Options, and add "corepolicy" option to use Bitcoin Core defaults
• Include Tonal digits and BTC symbol in bundled font