Chia plots check proofs

Chia-Plot-Status

GUI Tool for beginners and experts to Monitor and Analyse Chia Plotting log files, show health and progress of running plots and estimated time to completion. No setup, configuration or installation of python or whatever required. Just install and enjoy.

Features

• Monitor Progress of running plots
• Show estimated time to completion based on your already finished plots best matching your current plot config
• Monitor Health of plotting processes
• Already compatible with madMAx43v3r/chia-plotter (currently getting improved)
• Compatible with all plotters and plotting managers that use or are based on the official chia plotter (see Troubleshooting if something does not work from the get go)
• Show important information from log file in easy to read table
• Multiple folders with log files supported (multiple plotting rigs, anyone?)
• Multiple plots per log file supported (plot create –num n)
• Export of readable or raw data to Json, Yaml and CSV

See Chia Plot Status in action:

On Upside Down Crypto (YouTube):

On Patro TV (YouTube):

How it works

Chia Plot Status observes the log folders you give it to monitor which can be local or connected via network. By this it supports monitoring multiple plotting rigs as you can access them on your desktop even if your plotting rigs are headless. It regulary checks for new Log files in those folders and analyses them.

On basis of finished plots it builds a local statistic (on your machine, no data is send anywhere or pulled from any cloud) and uses them to calculate ETA / time remaining and warning thresholds for the Health of your plotting processes.

Working with many distributed plotting rigs

Recommended way: Use sshfs (with sshfs-win for Windows) to securely mount the log dirs of your plotting rigs on your desktop via highly encrypted network connection, where it is your desktop that initiates the mount. This can be set up so that the desktop can only access the log dirs and only has read access. Even if you use remote plotting rigs that you access over the internet this is the most secure way and you most likely access your remote servers via ssh already.

Other Options: Mount the log folders of all rigs as network shares (via samba on linux) if all your plotting rigs are in the local network or connected via VPN. Another way would be to make a cronjob on your plotting rigs that uses scp or rsync in append mode to copy the log dir to your desktop where you run Chia Plot Status, but if you can manage to set this up you should set up sshfs instead. Last, least preferred option: collect them with cloud apps like Google Drive (Chia Plot Status does not talk to any cloud services for you, you have to install those apps and mount your log folders in them yourself if you want to use them).

Best Practice:

• Only delete log files of finished plots if your hardware or the way you plot has significantly changed. Chia Plot Status uses finished plots to calculate ETA/Time Remaining as well as warning/error thresholds. If you delete finished log files the quality of those values decreases significantly.
• Use SSHFS to access the log directories of your plotting rigs
• Each plotting rig should have its own log folder, so they don’t mix and mess up estimates and warning thresholds for each other.
• Always log locally. If you log directly to a network share / NAS your plotting processes will crash if the connection becomes flaky. Prefer connecting your host machine (which runs Chia Plot Status) to networkshares on the plotting rigs, not the other way around.

Security / Trustworthiness

There are multiple attack vectors to consider:

1. The possibility that the core developer (me) is or becomes malicious

There is a saying: Where is a money trail, there is a way to sue/prosecute someone.

Chia Plot Status has buttons to donate via paypal both in the application itself and on the website.

Should the core developer (me) turn malicious, people could easily sue the core developer (me) and by that get the necessary details as in full name, address and day of birth, IBAN/Bic, everything from paypal.

If the core developer (me) becomes malicious this would be basically a how to get imprisoned speedrun (any %)

Even if you think you would not sue the core developer as he (me) might sit in a different country (germany), someone will as the Chia Plot Status Github Repository has between 2k to 4k visits daily and currently 24k downloads.

This should be more than enough to deter the core developer (me) from doing anything malicious.

2. The core developer (me) merges a pull request (code changes made by someone else) which contains malicious code without noticing

As seen on https://github.com/grayfallstown/Chia-Plot-Status/graphs/contributors there is only one other person who contributed a pull request so far and that wasn’t code but a documentation change.

The core developer (me) will check each pull request before merging as he (me) would have to run the code himself to check if the application works properly after merging that pull request and by that he (I) would get attacked by any malicious code that was contained in that pull request.

3. External Dependencies (as in libraries / code written by someone else) the application uses to do certain things (like to create the graphical user interface) become malicious.

Well, this is a tough one as even the core developer (me) has very little means to check external binaries for malicious code. The core developer (me) and every other developer using those libraries will get attacked by any malicious code in those libraries before they (we) distribute a new version of their (our) software containing that library to the users of their (our) softwares, as they (we) generally test their (our) applications before each release.

The core developer (me) takes the following precautions to mitigate that risk:

External dependencies are kept at a minimum to reduce this attack vector (chia-blockchain devs do the same)

Every release build is build on the same system and previously downloaded dependencies are never deleted/redownloaded. This prevents pulling in malicious code if the external dependency version used gets replaced with malicious code. But it also prevents reproducible builds that everyone can follow and reproduce step by step on their system, if the external dependency version actually does get changed. Well, this should raise concern anyway and in any case.

Updating Dependencies (external libraries / code written by someone else) is delayed (possibly indefinitely) until an update is required to implement a feature or to fix a bug. This gives anti virus providers time to determine if that library version is malicious, which would prevent an update.

Installation / Download

Windows: Download latest version You will get a blue warning saying this was published by an unknown developer.

Linux: First install dotnet 5.0 SDK, then either the Chia Plot Status deb or rpm package depending on your linux distribution (deb for ubuntu)

For Mac you currently have to build it.

Getting Log Files from PowerShell

The last part with 2>&1 | % ToString | Tee-Object writes the log to the PowerShell and to a file with a unique name for each plotting process.

Источник

[INFO] plots check — proofs overflow #4840

Comments

nerotiger commented May 13, 2021 •

Describe the bug
A clear and concise description of what the bug is. (If what you are experiencing is NOT a bug but instead a support issue, please open a Discussion instead!)

(venv) [x@y chia-blockchain]$ chia plots check

To Reproduce
Steps to reproduce the behavior:

run command «chia plots check» with/without -n 30 or 50
Proofs 56 / 50, 1.12
Proofs 39 / 30, 1.3

however, for the other plot, I have had a percentage under 100%

Expected behavior
A clear and concise description of what you expected to happen.
The check should return a proof value within 100%.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. Linux]
  Linux 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• OS Version/Flavor: [e.g. CentOS 7.2]
  CentOS Linux release 7.5.1804 (Core)
• CPU: [e.g. Intel Xeon 8175M]
  model name : Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz
  Additional context
  Add any other context about the problem here.
  (venv) [x@y

]$ python3 —version
Python 3.9.5

The text was updated successfully, but these errors were encountered:

Источник

How to validate your plots

Sh*t happens and chia isn’t an exception. If you want to make sure your plots are ok please continue reading. I had some problems before, client crashed, system crashed so I wanted validate what I have is in good shape. Surprisingly I had a few plots that did not pass the filter, so at the end I deleted them. I just had four thankfully.

Open PowerShell on Windows or on Linux go to the relevant chia directory where the chia binary is located

From there run the following that goes and checks all your plots

.\chia plots check

It will take time to go through all your plots, I did it live, so was plotting and haven’t run into any problems. At the end you will get something like:

2021-04-24T07:57:16.735 chia.plotting.check_plots : ERROR : Src size is incorrect error in getting challenge qualities for plot H:\plotk32-2021-03-01-09-07-foobar.plot
2021-04-24T07:57:16.735 chia.plotting.check_plots : ERROR Proofs 0 / 30, 0.0

At the end a nice summary report

2021-04-24T09:08:17.515 chia.plotting.check_plots : INFO Found 187 valid plots, total size 18.51065 TiB
2021-04-24T09:08:17.515 chia.plotting.check_plots : INFO 187 plots of size 32
2021-04-24T09:08:17.515 chia.plotting.check_plots : WARNING 4 invalid plots found:

What that means these plots are broken so you need to remove them. You can do that by just deleting these files from your disk. Make sure you refresh your plots after that – Refresh Plots at the Plots page.

Источник

Nothing tells you when you’re timing out on 30 second harvester proof checks #2651

Comments

coding-horror commented Apr 25, 2021 •

The problem

I thought I was farming, but I wasn’t — because something about my network caused the proof check to take more than the hard-coded 30 second limit.

I had an average time to win of 8 or 9 hours for more than 120 hours without a single win. This seemed statistically implausible, so I researched the logs, and cleared any errors or warnings in the logs (well done, all the warnings and errors in debug.log were indeed things I should fix!). Still no wins for a long time.

How to reproduce

Have a bunch of plots on slow storage media; when the proof check happens, verifying the proofs takes longer than the hard-coded 30 seconds allowed. You will never win a single Chia, but there’s absolutely nothing in the GUI to inform you that this is happening. You can view the logs, but in the logs it is not even presented as a warning (!), but as an INFO message:

Of the above, the proofs that take longer than 30 seconds are not eligible to win, but this is not logged as an ERROR or WARNING or surfaced in the UI in any way.

Expected behavior

The GUI will tell you «hey, your proof checks are too slow, there’s absolutely no chance for you to win, even if you are farming infinity plots»

Screenshots

Desktop

• OS: Windows 10
• OS Version/Flavor: latest 19042.928
• CPU: i7-1165G7

Additional context

I followed up on the #support channel in Keybase, where I got the important advice to enable INFO level logging and check for the 30 second proof limit.. and I wrote up a detailed account on the forum; if you need excruciating levels of detail, please check there 🙇‍♂️

Recommended solution

• Allow the 30 second proof check threshold to be configurable to account for slower storage (or relax it a bit)
• change the log level of beyond 30 second on harvester proof checks to WARN or ERROR
• in the logs, provide details on which files are being checked for proofs, so you can identify which media or NAS is potentially the problem one slowing everything down

The text was updated successfully, but these errors were encountered:

maomingyang1314 commented Apr 25, 2021

Moreover, I would like to know why your laptop can mount nearly 400TB disks, is your disk on nas?

maomingyang1314 commented Apr 25, 2021

Your NAS caused your task to time out. It is recommended to abandon this method.

AZ663 commented Apr 25, 2021

mariano54 commented Apr 25, 2021

We can’t change the consensus algorithm anymore, but we can change the log level, and show the files.

Please note that looking up qualities for plots passing the filter requires about 7 random reads in a plot, whereas actually looking up a proof requires 64 reads. It might not be feasible on a slow NAS, since these are sequential reads. Furthermore, you need to take into account network latency to propagate your proof and block to the network, so you should be under 5 seconds to reduce risk of losing rewards.

Actually, the proof of space library does them sequentially, but they could be done in parallel, since it’s a tree, so you could do 1 read, then 2, then 4, .. etc, for a total of around 7 sequential phases (one for each table in the plot). We haven’t got around to doing this yet.

mariano54 commented Apr 25, 2021

Also another thing to point out is that the responses are returned to the full node as they come out from the drives, so the high time is probably only affecting the slow drive or the slow NAS.

coding-horror commented Apr 25, 2021

Yeah, this only started happening as I added multiple NAS devices to the network. With 1 or 2 NASes, it was all fine. Once you get to 5.. not so much, especially if the algorithm picks plots on 6 different devices. At the very least

• definitely change the log level to ERROR (or WARN)
• surface this in the UI for sure, otherwise you’re silently «farming».. absolutely nothing. A silent killer.
• tell us WHICH files were picked for proofs so we can look for «problem» devices

immortalx commented Apr 26, 2021

Is there any benefit of using larger plot files in this case? Less files per directory in his case.

keliew commented Apr 26, 2021

I think the problem is inherent in how Windows manages mounted network drives.

If you’d to switch to a Linux-based farmer, it should work better. I know it’s not an option for most people, but it’d be a good test to see if that’s the cause.

kimklai commented Apr 26, 2021

I think lots of people need to know their whole system just too slow to provide the a valid answer in time. So please mark the logs as WARNING as they takes longer a certain threshold.

desek commented Apr 26, 2021

This was quite a rough finding since I’ve been happily farming on my RPi on wifi to a remote storage with proof checking usually between 60-90 seconds.

tartley commented Apr 26, 2021

Could some drives be powering down when idle?

bertybassett commented Apr 27, 2021 •

yeah my 1 Synology NAS is doing the same if I hit it hard with other services such as docker, Plex, etc

My Synology can sleep the HDs and out of the box I believe it set to that as default, no idea about yours.

Synology HDD hibernation
Synology NAS, expansion units, and USB HDDs enter different energy-saving modes after being idle. . Phase 1: HDDs power down after a period of inactivity (idle time). You may go to DSM > Control Panel > Hardware & Power > HDD Hibernation to configure the length of inactivity for the HDDs to enter hibernation.

cccat6 commented Apr 27, 2021 •

We can’t change the consensus algorithm anymore, but we can change the log level, and show the files.

Please note that looking up qualities for plots passing the filter requires about 7 random reads in a plot, whereas actually looking up a proof requires 64 reads. It might not be feasible on a slow NAS, since these are sequential reads. Furthermore, you need to take into account network latency to propagate your proof and block to the network, so you should be under 5 seconds to reduce risk of losing rewards.

Actually, the proof of space library does them sequentially, but they could be done in parallel, since it’s a tree, so you could do 1 read, then 2, then 4, .. etc, for a total of around 7 sequential phases (one for each table in the plot). We haven’t got around to doing this yet.

@mariano54
My disk array is not in the local but distributes in different places. I gather them as a huge disk and connect them with NFS. The latency of this solution for each random seek is between 200-1000ms. The average and the mean is about 400ms.
I wondering whether this is able to farm. 400ms*(7+64)seeks will be 28.4s. Just a little bit below the 30s limitation.
However, this solution support parallel read. If it is seeking 64 different places in the plot at the same time, it will be able to respond to all of them within a single seek time, which is about 400ms.
Is parallel seeking in the todo list and is that work as what I am thinking? When it will be updated?
Thanks a lot!

Источник