I was wondering if it would be possible to get sampling results without doing spark profiler --stop like you can do with timings? I want to be able to run the profiler for a long time in order to get accurate statistics and not having to stop the profiler every time I want to see the results.

Thanks!

This is technically possible, but it would require quite a lot of restructuring - and I'm not sure it's worth it for the functionality that would result.

In my opinion, it's fine to just wait a few minutes then fully stop to see results.

You shouldn't need to run the profiler for more than 2-3 minutes.

If you want more accurate results, I suggest increasing the sampling interval instead, using the --interval flag - the default is 4, maybe try 2 or 1! (it's measured in milliseconds)