Latency is simply how long it takes a request to go out through the proxy and come back with a response — lower is better, but how much it matters depends on what you're doing.
What causes high latency
Physical distance between you, the proxy, and the destination server adds unavoidable delay. Server load also matters a lot with free public proxies, since many people hitting the same server at once slows everyone's requests down.
When latency matters most
Real-time or interactive tasks — live browsing, anything with a tight timeout — suffer noticeably from high latency. Batch scraping jobs that run in the background are far more tolerant of slow individual requests since the work isn't time-sensitive.
A practical filter
Most proxy lists display latency for exactly this reason. Filtering to proxies under a few hundred milliseconds is a reasonable default for anything interactive, while background scraping jobs can generally tolerate much higher numbers.