Optimizing SpotCollector's Performance

When an incoming spot arrives from a spot source, SpotCollector displays the spot in the appropriate spot source window (unless the spot source is configured to display only announcement and talk messages), and starts looking through its Spot Database in reverse chronological order to see if there's already an Entry for the spotted station in the spotted mode near the spotted frequency. It keeps looking until

Thus, there are seven primary factors that most affect SpotCollector's performance and PC resource (CPU and RAM) consumption:

  1. the number of active Spot Sources
  2. the configuration of Spot Source windows to display or not display DX spots
  3. the size of the Spot Database
  4. the Max separation in time setting

  5. the rate of incoming DX spots
  6. web server
  7. hardware capabilities

Number of active Spot Sources

Collecting DX Spots from multiple sources is useful because any one source may provide only a subset of all DX Spots, and because individual sources can occasionally become unavailable or inaccessible. Within a minute or two, however, most of the DX Spots received from one Spot Source will exactly duplicate the DX Spots received from another Spot Source. To prevent redundant searches of the Spot Database, SpotCollector maintains a cache of recent DX Spots that it uses to filter out exact duplicates. While searching the cache for duplicates consumes far fewer PC resources than searching the Spot Database, it is not free. Thus reducing the number of active Spot Sources to the minimum that meets your needs will reduce the load on your PC. If your DXing needs are met by 4 Spot Sources, there is no need to connect to 6 just because SpotCollector has this capability.

Configuration of Spot Source windows to display or not display DX spots

On the Configuration window's Spot Sources tab, an Ann/Talk option is provided for each Spot Source. If this option is enabled, only announcement and talk messages are displayed and retained in Spot Source windows; DX spots are discarded after being entered into the Spot Database. If the Ann/Talk option is disabled, DX spots are retained. These retained spots consume memory; with many active Spot Sources generating DX spots over multiple days, this memory consumption can be signficant enough to degrade overall system performance. Since all incoming DX spots are accessible via the Spot Database Display (except those rejected as too old by the Maximum age of valid incoming spots setting, and those rejected by the by the Special Callsigns mechanism), there is no compelling reason to display them in Spot Source windows. For optimum performance, then, check all the Ann/Talk boxes on the Configuration window's Spot Sources tab, and restart SpotCollector; note that the IRC spot source's Ann/Talk option is permanently enabled.

Size of the Spot Database

By retaining Spot Database Entries, one can observe propagation patterns over time, and discover a needed DX station's operating habits. If you aren't concerned about resource consumption, the only upper bound on the size of the Spot Database is free space on the hard drive that hosts it. However, a larger Spot Database causes SpotCollector's database engine to consume more RAM; as free RAM becomes scarce, Windows spends more and more CPU time shuttling information between RAM and your hard drive so that applications have the RAM they need when they're running (a scheme referred to as virtual memory). Thus there is a direct tradeoff between Spot Database size and PC resource consumption. You can configure SpotCollector to automatically prune away the oldest Entries in your Spot Database in order to maintain a maximum size -- either hourly, or each time SpotCollector starts. To do this, see the Size Limit Panel section in SpotCollector's documentation. If you are observing long-term propagation patterns or staking out a needed DX station, then maintaining a large Spot Database makes sense; if not, you can reduce PC resource consumption by configuring SpotCollector to automatically prune Entries older than a day or two. If you run SpotCollector 24x7, then enable hourly pruning; if you start SpotCollector at the beginning of a DXing session, then pruning on startup will be more efficient. During contests, consider temporarily reducing the Prune entries older than this age parameter, as even a setting of 2 days can retain 20 thousand Spot Database Entries.

You can configure SpotCollector to limit the Spot Database to a relatively small size (e.g. 1 day) but still retain unlimited Spot Database Entries for specific DX stations whose operating patterns you're trying to discern: add the callsigns of such stations to the Special Callsign list with a NoPrune tag.

Max separation in time setting

The Max separation in time setting in the Combination Criteria panel on the Configuration window's Spot Database tab determines how far back in time SpotCollector searches the Spot Database for an existing Entry when processing an incoming DX spot. Thus larger values of Max separation in time mean that SpotCollector could review a lot of Spot Database Entries each time a new DX spot arrives -- and thus consume more PC resources. Smaller values of Max separation in time reduce PC resource consumption, but could produce multiple Spot Database Entries for the same DX station, which is much less a resource consumption issue than a usability issue: were you to set Max separation in time to 0, then the Spot Database Display would contain as many redundent Entries as a raw telnet cluster. A good baseline for Max separation in time is 60 (minutes). During non-contest conditions, this minimizes redundant Entries while limiting the Spot Database searching to 50-100 entries for each incoming DX spot. Reducing Max separation in time to 30 (minutes) might be appropriate during contests that generate high DX spot volume.

Rate of incoming DX spots

During contests, the rate at which DX spots arrive increases dramatically above the normal baseline of 50-100 spots per hour to peaks exceeding 500 spots per hour. Furthermore, larger numbers of DX stations can be active, so more new Spot Database Entries are created per hour, which means that the average Spot Database search is longer and thus consumes more PC resources. Spot sources connected to CW, RTTY, and FT8 skimmers can generate high spot volumes in the absence of contests; the Reverse Beacon Network can generate 60,000 spots per hour during a popular contest! Since you have no control over the incoming DX spot rate during contests, consider compensating for it by adjusting one or more of the above three factors.

Process Priority

You can increase the fraction of your CPU's attention that is devoted to SpotCollector by selecting Above Normal or High in the Process Priority panel on the Configuration window's Spot Database tab. Note that doing this will reduce the fraction of your CPU's attention that is devoted to other competing applications.

Web Server

SpotCollector's built-in web server lets you view spots from any PC on your network. If you're not using this capability, you can save both RAM and processing time by disabling the web server on the Configuration window's Spot Database tab

Hardware capabilities

SpotCollector and DXKeeper both employ a database. In both applications, most database accesses are read operations, which are faster and less resource consumptive than write operations. You can generally improve the performance of database systems by

If your PC's processor provides more than two cores, it's been observed that SpotCollector performance continues to improve with up to 3 dedicated cores. Instructions for dedicating a core to an application can be found in

http://www.addictivetips.com/windows-tips/how-to-set-processor-affinity-to-an-application-in-windows

Process Lasso is utility that provides automation for dedicating cores to application.

When running a non-trivial application like SpotCollector or DXKeeper, there is always a bottleneck that limits performance. Determining which of the above hardware upgrades will actually improve performance is difficult without instrumentation and/or monitoring gear. Sometimes, a change yields a significant improvement in the performance of one application, but yields no benefit at all to another application. And sometimes, a change overcomes an application's current bottleneck, but performance only increases a few percent before an entirely different bottleneck is encountered.

Thus the general rule is take the cheapest steps first. Optimizing SpotCollector's settings as described above costs nothing, but can yield major benefits. If you already have a quad core CPU, it costs nothing to assign two of its cores to SpotCollector. After that, additional RAM is usually the right next course of action.

Running SpotCollector on Windows Vista, 7, 8, and 10

By default, Windows Vista, Windows 7, Windows 8, and Windows 10 enable "visual effects" that can on some systems result in SpotCollector "stealing" the mouse cursor. This can be frustrating if you're trying to compose an email message or update a document, only to have the text you're typing suddenly begin appearing in the Filter panel on SpotCollector's Main window. You can configure Windows to prevent this.

Mouse cursor "stealing" can also occur if SpotCollector spends so much time searching its Spot Database that Windows erroneously concludes that SpotCollector must not be functioning correctly, and temporarily displays a not responding message in SpotCollector's title bar. If you see this happening, either reduce the size of the Spot Database as described above, or reduce the Max separation in time setting as described above.


Post a question or suggestion on the DXLab Discussion Group

Getting Started with Spot Collection

Getting Started with DXLab

OptimizeSCPerformance (last edited 2019-03-17 19:26:47 by AA6YQ)