== Getting Started with Spot Collection, Active DX Identification, and Analytics == During the late 1980s and early 1990s, VHF and UHF spotting networks began moving from voice repeaters to VHF and UHF [[http://www.dxcluster.info/|PacketClusters]] that exploited personal computers. In the mid 1990s, the Internet's [[http://en.wikipedia.org/wiki/Telnet|Telnet]] protocol was used to construct a [[http://www.dxcluster.info/telnet/index.php|network of DXClusters]], making it easy for any DXer to monitor ''spots'' -- reports of active DX stations -- on his or her Internet-connected computer. Being aware of DX stations that are currently active or have recently been active, on what bands, and in what modes provides valuable insight to the modern DXer, particularly if the locations from which a DX station was heard are known: * actual (as opposed to forecast) band openings are revealed * operating patterns of specific DX stations can be deduced, increasing the probability of making a future QSO Identifying a currently active DX station as being ''needed'' for one or more awards the DXer is pursuing provides the opportunity to make a QSO that will advance award progress. Viewing raw spots in a scrolling window as they arrive from a '''!PacketCluster''' or '''DXCluster''' or '''WSJT-X''' requires the DXer to perform the above analyses - a challenging task even with a single source of spots and one or two award objectives. !SpotCollector ingests those raw spots and processes them in real time to provide continuous answers to these questions: * For DXers seeking QSOs with stations in many zones, entities, states, and islands, and for DX Marathon participants early in the year * what stations are currently active, on what bands, and in what modes? * from what locations is each active station being spotted? * which active stations are ''needed'' for any of the awards the DXer is pursuing? * what bands are currently open from the DXer's QTH, and to what continents? * what bands have been open from the DXer's QTH at what times in the past, and to what continents? * For DXers pursuing stations that a rarely active * what's been the activity pattern for a station I'd like to work? * when and on what bands will there be propagation openings to a station I'd like to work? To accomplish this, '''!SpotCollector''' captures spots from multiple '''spot sources''': * up to four telnet-accessible '''DXClusters''' or skimmers * the '''DX Summit''' spotting network, by way of the #CQDX IRC channel * a local '''!PacketCluster''', by way of a Terminal Node Controller (TNC) attached to one of your PC's serial ports * one or more local WSJT-X instances Collecting spots from multiple DX cluster nodes is useful because some spot sources report DX from some geographic areas faster than from other areas, and because individual sources can occasionally become unavailable or inaccessible. A local instance of [[GettingStartedwithK1JTModesDirect|WSJT-X]] is an excellent source of spots. Some DX Cluster nodes can be configured to convey spots from the [[CollectingSpotsRBN|Remote Beacon Network (RBN)]], which provides a high volume of spots - several per second - from a network of automated Skimmers. !SpotCollector combines '''DX Spots''' for the same station/frequency/mode and stores them in a '''Database of Active DX Stations''' on your PC that can be simultaneously displayed in a table, on a world map, on a bandspread display, on a Spectrum-Waterfall display, and to reveal actual propagation: {{http://www.dxlabsuite.com/Wiki/Graphics/SpotCollector/MultipleViewsNoTitle.jpg}} By using spots to track active DX stations, the DXer can see current band openings from his or her QTH, observe propagation patterns over time, and discover the operating habits of ''needed'' DX stations. If you are using '''DXKeeper''' to log QSOs and track progress against DXing awards, !SpotCollector will use '''font color''' to identify active DX stations that are ''needed'' for one or more of the DXCC, IOTA, Leaderboard, Marathon, VUCC, WAS, WAZ, or WPX awards you're pursuing, as specified in your [[DXingObjectives|DXing objectives]]; when you work or confirm a ''needed'' DX station, !SpotCollector automatically updates the information it displays to reflect the new status. === Managing Spot Sources === On installation, !SpotCollector comes preconfigured with connection information for several spot sources, but you must [[ManagingSpotSources|specify a username and password before a spot source can be enabled]] to supply spots. You can also [[ManagingSpotSources|select a different set of spot sources]] more appropriate to you needs. While all spot sources eventually receive all spots, spots sources closer to a DX station can notify you of that DX station's presence as much as 60 seconds earlier than spot sources located further away. DXers should consider selecting spot sources in Asia, Europe, North America, and South America. === Filering and Pre-Filtering === By default, !SpotCollector inspects each incoming spot of a station operating in a particular mode near a particular frequency, discards it if is [[https://www.dxlabsuite.com/spotcollector/Help/ConfigurationDatabase.htm#Maximum%20age%20of%20valid%20incoming%20spots|too old]], and otherwise uses the information from the spot to either update an existing Spot Database Entry for that station in that mode near that frequency, or to create a new Spot Database Entry. !SpotCollector's [[SpotDatabaseFilter|Filtering]] mechanism lets you focus on a specified subset of the information contained in the Spot Database, like "all active CW stations on the transceiver's current band with whom a confirmed QSO would advance award progress". Filtering does not modify the content of the Spot Database; it modifies only your '''view''' of that data - a view you can quickly change to meet your need for information. In contrast, !SpotCollector's [[https://www.dxlabsuite.com/spotcollector/Help/Prefiltering.htm|Pre-filtering mechanism]] discards incoming spots that meet criteria you specify without extracting any information from them. When using this mechanism, take care to not discard spots that could identify useful propagation openings or reveal the operating patterns of needed DX stations. === Determining the Mode Being Used by a Spotted Station === DX spots specify a callsign, frequency, date, time, and (optionally) up to 29 characters of "notes". Given the absence of an explicit mode designator, !SpotCollector first checks an incoming spot's notes for the presence of a mode designator; if one is present, the Spot Database Entry created will specify the mode designated in the spot notes. If no mode designator is specified in the spot's notes, then !SpotCollector queries the [[https://www.dxlabsuite.com/spotcollector/Help/ConfigurationGeneral.htm#Sub-band%20Definition%20panel|specified Sub-band Definition file]] for an entry whose frequency range encompasses the incoming spot's frequency; if an entry is found, the mode specified in that entry is used in Spot Database Entry created from the spot. After initial installation, !SpotCollector references a Sub-band Definition file named {{{BandModes.txt}}} that defines sub-bands from 160m to 12cm for CW, LSB, USB, FM, and RTTY. You can modify or extend the entries in this file to reflect local usage and to include sub-bands for additional digital modes. A Sub-band Definition file named {{{BandModes 2022-12-06.txt}}} is also provided in !SpotCollector's folder; it defines sub-bands from 2190m to 12cm for CW, LSB, USB, FM, RTTY, PSK31, JT65, JT9, FT8, and FT4. To utilize this Sub-band Definition file, in the '''Sub-band Definition''' panel at the bottom of the Configuration window's '''General''' tab, 1. click the '''Select''' button 1. use the '''Open''' window that appears to navigate to your {{{SpotCollector}}} folder and select the {{{BandModes 2022-12-06.txt}}} file 1. terminate and restart !SpotCollector After performing the above steps, the modes of ''newly-received'' spots will be determined using the sub-band definitions in {{{BandModes 2022-12-06.txt}}} === Determining the Location of each Active DX Station === To enable !SpotCollector to obtain accurate location information for each active DX station (grid square, IOTA tag, CQ zone, ITU zone, continent, primary administrative subdivision, region), 1. if DXView has not been installed, [[InstallApplication| direct the DXLab Launcher to install it]] 1. direct DXView to download the current versions of the [[InstallDXCCDatabase|DXCC database]] and [[InstallUSAPDatabase|USAP database]] 1. check the '''Lookup missing location info''' box on !SpotCollector's '''Configuration''' window's '''General''' tab so that !SpotCollector will perform a lookup on active callsigns in the '''DXCC''' or '''USAP''' database === Displaying a Tabular View of Active DX Stations === The '''Spot Database Display''' in !SpotCollector's main window is a scrollable grid with entries for each DX station operating near a particular frequency in a particular mode. '''Each Spot Database Entry''' [[http://www.dxlabsuite.com/spotcollector/Help/SpotDatabase.htm#Spot%20Database%20Entry|includes]] * the DX station's '''callsign''' * the '''frequency''' on which the DX station was most recently spotted * the '''mode''' being used by the DX station (if not specified in the spot notes, [[SpotDatabaseMode|inferred]] based on frequency and the current [[SubbandDefinitions|Sub-band Definition file]] * the '''first time''' (in UTC) at which the DX station was spotted near this frequency in this mode * the '''last time''' (in UTC) at which the DX station was spotted on this frequency in this mode * the most recent '''notes''' accompanying a spot of this DX station Spot Database Entries are automatically constructed and updated as incoming DX spots arrive at from spot sources like DX Central. Unless the bands are dead, you should already see a few entries. You can [[SpotDatabaseFormat|format]] the Spot Database Display to present information that helps you understand the behavior of needed DX stations and identify band openings from your QTH: {{https://www.dxlabsuite.com/Wiki/Graphics/SpotCollector/SpotDatabaseDisplayAnnotated.jpg}} Armed with this knowledge, you can focus your listening on times and frequencies more likely to be productive. You can [[SpotDatabaseSort|sort]] Spot Database Entries, and you can [[SpotDatabaseFilter|filter]] the Spot Database Display to show only entries matching a particular callsign, DXCC entity, band, mode, continent, origin (location of the stations that spotted the DX) or combinations thereof. There are several ways to [[DetermineSpottingStationLocations|determine from where an Active DX station is being spotted]]. ==== Identifying Active Stations With Whom a Confirmed QSO Would Advance Your Progress Towards the Awards You Are Pursuing ==== If you are using DXKeeper to [[Logging|log your QSOs]], then the font used to render Spot Database entries will be colored to indicate the impact of a QSO with the spotted station with respect to your [[http://www.dxlabsuite.com/dxlabwiki/DXingObjectives|DXing objectives]] : * red: a QSO with this station will give you a new DXCC entity/entity-band/entity-mode, or a new IOTA island group, or a new Leaderboard band/mode, or a new Marathon country/country-band/country-mode/zone/zone-band/zone-mode, or a new VUCC grid-band, or a new WAS state/state-band/state-mode, or a new WAZ zone/zone-band/zone-mode/zone-band-mode, or a new WPX prefix/prefix-band/prefix-mode * blue: a QSO with this station will give you a new way of confirming a worked but still uncomfirmed DXCC entity/entity-band/entity-mode, IOTA island group, VUCC grid-band, WAS state/state-band/state-mode, WAZ zone/zone-band/zone-mode/zone-band-mode, or WPX prefix/prefix-band/prefix-mode * black: a QSO with this station will not advance your progress You can change these font colors by using the '''Font Colors''' panel on the Configuration window's '''Spot Database Display''' tab. You can [[ChangingFrequencyBandspread|direct Commander to display a Bandspread Window]] showing active DX Stations with this color-coding. ==== Identifying Active Stations Known to Participate in eQSL.cc and/or LoTW ==== If [[https://www.dxlabsuite.com/dxview/|DXView]] is installed and has been directed to [[https://www.dxlabsuite.com/dxview/Help/Configuration.htm#Databases%20tab|download current eQSL and LoTW Databases]], the background color of each Spot Database Entry will indicate whether the Entry's callsign is known to participate in eQSL.cc and/or ARRL's Logbook of the World (LoTW): * pink: the Entry's station is an Authenticity-Guaranteed (AG) participant in eQSL.cc * yellow: the Entry's station has submitted QSOs to LoTW * light blue: the Entry's station is an Authenticity-Guaranteed (AG) participant in eQSL.cc and has submitted QSOs to LoTW You can change these background colors by using the '''Background Colors''' panel on the Configuration window's '''Spot Database Display''' tab. ==== Determining Why a Spot Database Entry's Callsign is Considered ''Needed'' ==== There are several ways to [[SpotDatabaseNeedDetermination|determine why a Spot Database Entry's active Station is needed]], meaning that a QSO with that station would advance your progress towards the DXCC, IOTA, Marathon, WAS, WAZ, and WPX awards ==== Selecting a Spot Database Entry ==== Clicking a Spot Database Entry will convey the Entry's callsign to [[https://www.dxlabsuite.com/dxview/|DXView]], which will display information about the DX station's location -- latitude and longitude, beam heading, distance, local time, CQ zone, and ITU zone -- and highlight that location on a world map with a blue dot connected to your QTH by the signal path; DXView will also convey the location information to [[https://www.dxlabsuite.com/propview/|PropView]]. Double-clicking a Spot Database Entry will convey the entry's callsign, frequency, mode, and grid square to [[TransceiverControl|Commander]], [[Logging|DXKeeper]], [[DxInfo|DXView]], [[Propagation|PropView]], and/or [[QSLRouteDiscovery|Pathfinder]], enabling them to [[TransceiverControl|QSY your transceiver]] to the appropriate frequency and mode, [[SCRotateAntenna|rotate your antenna]], display [[DxInfo|DX information]], prepare to [[Logging|log a QSO]], prepare to [[Propagation|predict propagation]], and/or prepare to [[QSLRouteDiscovery|seek QSL information]]. If [[Operation|WinWarbler]] is running, * With the '''Digital Mode Application''' panel's '''Name''' set to ''!WinWarbler'' on the Configuration window's '''General''' tab, double-clicking a Spot Database Entry whose mode is ''PSK31'', ''PSK63'', ''PSK125'', or ''RTTY'' will convey the entry's callsign, frequency, mode, and grid square to !WinWarbler, which will populate its '''QSO Info''' panel with this information and direct [[TransceiverControl|Commander]] to QSY your transceiver to the appropriate frequency and mode. Configure !WinWarbler's '''Xcvr Mode''', '''Mark Offset''', and '''Optimal Offset''' settings as described [[ConfigureRTTY|here]] so that double-clicking on the callsign of a RTTY station will QSY the transceiver to the appropriate frequency for immediate decoding (if the station's Mark frequency was accurately spotted). * Double-clicking a Spot Database Entry whose mode is ''CW'', ''SSB'', ''AM'', or ''FM'' will only convey the entry's callsign, frequency, mode, and grid square to !WinWarbler if that mode's box is checked in the '''Service''' column in the '''Actions with Digital Mode Application connection''' panel on the '''General''' tab of !SpotCollector's '''Configuration''' window * If you are using [[GettingStartedwithK1JTModesDirect|Direct Interoperation with WSJT-X]], have !SpotCollector connected to at least one instance of WSJT-X, and have DXKeeper running, double-clicking a Spot Database Entrywhose Mode field is a K1JT mode (FST4, FT4, FT8, JT4, JT9, JT65, JT9, MSK144, Q65, QRA64, and WSPR) for which the [[https://www.dxlabsuite.com/spotcollector/Help/ConfigurationGeneral.htm#Sub-band%20Definition%20panel|sub-band definition file]] contains an entry for that K1JT mode will set the transceiver's frequency to bottom of that sub-band and set the transceiver's mode to that specified in the '''Transceiver mode for WSJT-X''' panel on the Configuration window's '''General''' tab === Enabling Audio, Email, and Text Announcements === * [[AudioAnnouncements|Configuring SpotCollector to audibly announce the creation of new Spot Database Entries for needed DX stations]] * [[eMailAnnouncements|Configuring SpotCollector to send email or SMS messages announcing the creation of new Spot Database Entries for needed DX stations]] === Displaying a World Map View of Active DX Stations === * [[ActiveDXPlotting|On DXView's World Map]] * [[ActiveDXPlottingDXAtlas|On the DX Atlas World Map]] * [[GoogleEarthPlotting|On Google Earth]] === Displaying a Bandspread View of Active DX Stations === You can display active DX stations on * [[ChangingFrequencyBandspread|Commander's vertically-oriented Bandspread window]] * [[SpectrumWaterfallIcom|Commander's horizontally-oriented Spectrum-Waterfall window, driven by an Elecraft K4 or an Icom IC-7300, IC-7610, IC-7850, IC-7851, or IC-9700]] * [[FlexSignatureDisplayCallsigns|FlexRadio's SmartSDR Panadapter, driven by a FlexRadio Signature transceiver]] === Displaying a Propagation View of Active DX Stations === !SpotCollector can use reports of active DX stations by stations near your QTH to [[DeterminingActualPropagationFromSpots|display activity on each band by time of day]], and to [[DeterminingActualPropagationFromSpots|display activity on each band by continent]]. === Interoperating with WSJT-X === !SpotCollector can [[GettingStartedwithK1JTModesDirect|directly interoperate with WSJT-X]] * use WSJT-X as a spot source * highlight callsigns displayed in the WSJT-X '''Band Activity''' panel with whom a confirmed QSO would advance your progress towards the DXCC, IOTA, Marathon, WAS, WAZ, and WPX awards given your [[http://www.dxlabsuite.com/dxlabwiki/DXingObjectives|DXing objectives]] * identify callsigns displayed in the WSJT-X '''Band Activity''' panel that participate in eQSL-AG, LoTW, or both * convey QSOs logged in WSJT-X to DXKeeper for logging === WWV Spots === Many '''!PacketClusters''' and '''DXClusters''' also convey '''WWV spots''' -- reports from the United States National Institute of Standard's [[http://en.wikipedia.org/wiki/WWV_%28radio_station%29|radio station WWV]] that include information relevant to HF radio propagation: the Solar Flux Index ('''SFI'''), the Boulder '''A'''-index, and the Boulder''' K'''-index. Information from the most recent WWV spot is displayed in the Main window's '''WWV''' panel. !SpotCollector records the information from each WWV spot, and presents it in a graphical view illustrating the behavior of these indices over the most recent 31 days. Clicking the '''History''' button in the '''WWV''' panel in the upper left corner of !SpotCollector's Main window displays the '''Solary & Geogmagnetic Parameters''' window: {{http://www.dxlabsuite.com/Wiki/Graphics/SpotCollector/SolarGeomag.jpg}} You can examine each of the three indices individually by checking or unchecking the '''SFI''', '''A''', and '''K''' boxes in the lower corners of the display. Given the sun's 27-day rotation period, this information can also be helpful in predicting future HF propagation. !SpotCollector automatically conveys this information to [[Propagation|PropView]]. ==== Enabling a DX Cluster's WWV Spot Reporting ==== To enable WWV spots to DXClusters controlled by [[http://www.dxcluster.org/main/|DX Spider]] or [[http://bcdxc.org/ve7cc/ccc/CCC_Commands.htm|CC Cluster]] applications, issue the command set/wwv To enable WWV spots to DXClusters controlled by the [[http://www.nc7j.com/arcluster/arusermanual-6/ARC6%20Command%20Summary.html|AR-Cluster]] application, issue the command set/wwv/output on ==== Determining Whether a DX Cluster Conveys WWV Spots ==== To determine whether a DX Cluster conveys '''WWV spots''', 1. connect to the DX Cluster 1. In the '''Telnet DXClusters''' panel on the '''Spot Sources''' tab of !SpotCollector's Configuration window, uncheck the DX Cluster's '''Hide''' and '''Ann/Talk''' boxes 1. A window will appear that enables you to send commands to the DX Cluster and see the responses; issue the ''SH/WWV'' command and then strike the '''Enter''' key; if the DX Cluster responds by displaying previous '''WWV spots''', you can expect it to report future '''WWV spots''' ==== Automatically Updating the Spot Database and WWV History on Startup ==== To configure a DX Cluster spot source to update your Spot Database with the most recent 100 DX spots and the most recent 36 WWV spots, 1. click the''' Config''' button to open !SpotCollector's Configuration window and select the '''Spot Sources''' tab. 1. in the '''Initial Cluster commands''' panel, a. paste this command into the spot source's '''Command''' box: ''SH/WWV/36<13><10>SH/DX/100<13><10>set/dxgrid'' a. check the '''Enable''' box to the left of the '''Command''' box Now when !SpotCollector starts up, it will direct that cluster to report the most recent 100 DX spots and the most recent 36 WWV spots. You can configure S!potCollector to send initial cluster commands to multiple spot sources. === Spot Sources === * [[http://www.dxlabsuite.com/spotcollector/Help/SourceConnection.htm#Connecting to DXClusters|DX Clusters]] * [[http://www.dxcluster.info/telnet/index.php|Worldwide DX Cluster Directory]] * [[https://ng3k.com/Misc/cluster.html|NG3K Telnet Access to DX Clusters and Reverse Beacon Network (RBN)]] * [[CWSkimmer|CW Skimmer]] * [[http://www.dxlabsuite.com/spotcollector/Help/SourceConnection.htm#Connecting to the #CQDX IRC channel|DX Summit Web Cluster]] * [[GettingStartedwithK1JTModesWithJTAlert|JT-Alert]] * [[http://www.dxlabsuite.com/spotcollector/Help/PacketClusterInteraction.htm|Packet Cluster]] * POTA spot sources * dx.cqspot.com, port 1234 * [[http://www.dxlabsuite.com/RBN/index.htm|Reverse Beacon Network]] * SOTA spot sources * cluster.sota.org.uk port 7300 * elgur.crabdance.com port 7300 * [[GettingStartedwithK1JTModesDirect|WJST-X]] === Using the Spot Database Display === * [[SpotDatabaseFormat|Formatting the Spot Database Display]] * [[SpotDatabaseNavigate|Navigating the Spot Database Display]] * [[SpotDatabaseSort|Sorting the Spot Database Display]] * [[SpotDatabaseFilter|Filtering the Spot Database Display]] * [[SpotDatabasePrefilter|Pre-filtering the Spot Database Display]] * [[FilteringPrefilteringClusterFilter|Spot Source Filtering vs. Pre-filtering vs. Spot Database Display filtering]] * [[SpotDatabaseMode|Determining a DX Spot's Mode]] * [[SpotDatabaseNeedDetermination|Determining why a Spot Database Entry is Needed]] * [[DetermineSpottingStationLocations|Determining From Where an Active DX Station is Being Spotted]] * [[SpotDatabasePropagationPrediction|Predicting Propagation for a Spot Database Entry's DX Station]] * [[SpotDatabaseDXGridSource|Determining the Source of a Spot Database Entry's DX Station's Location]] * [[SpotCollectorWebServer|Viewing the Spot Database Display from Another Computer, Tablet, or Smart Phone]] === Callsigns Requiring Special Action === * [[CreateEntityOverride| Using Overrides to Deal with Ambiguous or Non-compliant Callsigns]] * [[IdentifyLogClubMembers|Identifying and Logging QSOs with Club Members, Callsigns with Leaderboard Competitions, and other Special Callsigns]] * [[IdentifyBadCallsigns|Identifying Callsigns that when spotted should be ignored or whose spots should be ignored]] === Additional Functionality === * [[GeneratingSpots|Posting DX Spots]] * [[AnnouncementsAndMessages| Viewing Announcements and Talk Messages]] * [[IncludingUSStatesInSpotDatabaseEntries|Including US States in Spot Database Entries]] * [[DeterminingActualPropagationFromSpots|Analyzing Propagation]] * [[HidingDXYouCantHear|Hiding Active DX Stations That You Probably Can't Hear]] * [[NeededDXFromSkimmer|Displaying Needed DX Stations Heard by a Skimmer]] * [[GettingStartedwithK1JTModesDirect|Directly Interoperating with WSJT-X]] * [[DXCCUserSelectedDigitalMode|Tracking DXCC Award Progress for a User-selected Digital Mode]] * [[IgnoringSpots|Ignoring Spots by Band, Mode Family, or Origin]] * [[PursuingEntityBandModes|Pursuing Unworked and Unconfirmed Entity-Band-Modes]] === Configuration === * [[WinWarblerActivationOnCWSSBSpotDatabaseEntryDoubleClick|Configuring SpotCollector to activate WinWarbler when a CW or SSB Spot Database Entry is Double-clicked]] * [[SpotDatabaseDisplayAzimuth|Enabling SpotCollector to compute and display an azimuth for each active DX station]] * [[SubbandDefinitions|Defining Sub-bands]] * [[SupportNewMode|Adding Support for a New Mode]] === Improving Performance === * [[SpotDatabaseEntriesHaveWrongModes|Spot Database Entries Have Incorrect Modes]] * [[OptimizeSCPerformance|Optimizing SpotCollector's Performance]] * [[OptimizeWindowsForSpotCollector|Optimizing Windows Vista, 7 and 8 for SpotCollector]] * [[OptimizeWindows10ForSpotCollector|Optimizing Windows 10 and 11 for SpotCollector]] === Troubleshooting === * [[EmptyDatabase|No entries in your Spot Database display?]] * [[DXSummitConnection|Can't connect to DX Summit via IRC?]] * [[OptimizeWindowsForSpotCollector|SpotCollector stealing mouse cursor focus?]] ---- [[DXLabDiscussionGroup|Post a question or suggestion on the DXLab Discussion Group]] [[GettingStarted|Getting Started with DXLab]]