Intel had in development a "rugged NUC" with the Atom-branded equivalent of the Intel N100 CPU, which supported ECC and which was expected to have a low price.
Unfortunately, I assume that this product has been canceled a few months ago, when Intel sold their NUC business to ASUS.
There are a few small computers that support ECC and which use obsolete Intel Tiger Lake or Tremont-core-based Intel Elkhart Lake CPUs, but those CPUs are a dead end, being slow and supporting instruction sets that are different from the current mainline Intel CPUs, so I would not recommend any of them.
The best remaining choice depends on which is more important, the size and the power consumption or the price of the server.
For very small size and low power consumption I am not aware of any good solution at a reasonable price, because even when some of the Arm CPU SoCs or Intel or AMD mobile CPUs support ECC, I have not seen any such computer board that includes the ECC support. There are some industrial computers with ECC, but those are expensive for what they offer.
If only the cost is the problem, and second-hand servers are avoided because the server to be bought is intended to be used for many years, then a server with desktop Intel or AMD CPUs must be used. The MBs with the Intel W480 chipset are expensive, so the cheapest solution is to use one of the AM5 MBs that specify ECC memory support, e.g. from ASUS or ASRock Rack, together with one of the cheaper Ryzen 7000.
Another option is an older AM4 MB, like the Mini-ITX ASRock Rack X570D4I-2T ($400 due to including dual 10 Gb/s Ethernet ports), which has the advantage of using cheaper older Ryzen 5000 CPUs, with cheaper DDR4 ECC memory, so the total system cost would be reasonable.
The only disadvantage of the desktop Ryzen CPUs when used as servers is that, even if they have excellent energy efficiency when they are actually running programs, they have a relatively high idle power consumption, because only the cores are shut down when doing nothing, while the I/O die has a permanent consumption around 20 W or more. Therefore one must choose between the low idle power consumption of a few watts of the laptop CPUs and the ECC memory support of the desktop CPUs.
Because in my home lab most servers alternate between times when they are used intensively with times when they stay idle for hours or days, except for one server that is connected permanently to the Internet, all the others are used with Wake-on-LAN, so they are shut down when idle, for negligible power consumption.
