.. highlight:: rst .. _scart: ##### scart ##### **Import/export MiniSEED data to/from SDS archives.** Description =========== The archive tool scart reads and writes :term:`SDS` archives and files in miniSEED format and checks miniSEED archives or prints stream information. * **Dump mode:** Create miniSEED files (multiplexed), e.g. for playbacks, from :term:`SDS` structured data (e.g. created by slarchive). * **Dump mode:** Play back records directly out of an SDS structure. * **Import mode:** Import multiplexed miniSEED files into a local SDS waveform archive. * **Import mode:** Import data using the :ref:`global_recordstream` interface into a local SDS waveform archive. * **Import mode:** Read data from any :ref:`global_recordstream` interface and dump it to file. * **Check mode:** Check an archive of miniSEED files for out-of-order records in files. .. warning:: * When creating :term:`SDS` archives, scart simply appends the new records to existing ones. Multiple imports of the same data result in duplication. * Out-of-order imports of waveforms into a SDS archive result in out-of-order records which may not be processed. Clean your archive using :ref:`scmssort`. * **Before importing miniSEED data** into an SDS archive they must be sorted by time and duplicate records must be removed. Otherwise, the SDS archive may not be correctly readable by other modules. Therefore, combine scart with :ref:`scmssort` for multiplexing and removal of duplicates. .. hint:: In dump and import mode output streams may be filtered by * Time windows (:option:`-t`), * Network-station-location-channel (NSLC) lists (:option:`--nslc`) created, e.g., with * :ref:`scinv` from inventories, * :ref:`scdumpcfg` from bindings configuration, * scart itself from other miniSEED files or achives. In dump mode output streams may also be filtered by * Time window - stream lists (:option:`--list`, dump mode) generated by :ref:`scevtstreams` for particular events. .. _scart-config: Configuration ============= scart can make use of :ref:`global_recordstream` implementations which are provided by additional plugins. For loading additional plugins, e.g. the *xyz* plugin create and configure :file:`scart.cfg`: .. code-block:: sh plugins = xyz Examples ======== .. hint:: The usage of wildcards in place of network, station, location or channel code is allowed in many options (:option:`-n`, :option:`-c`, :option:`-l`, :option:`--list`, :option:`--nslc`) and follows these rules: * Import mode: the wildcards are passed to the :ref:`global_recordstream` interface, that interprets them. Normally both "*" and "?" are supported by RecordStreams. * Dump mode: the wildcards are interpreted by scart command that supports "*" for network, station, location codes and "*", "?", "(", ")", "|" for channel code. #. Extract data from the default :term:`SDS` archive in :file:`$SEISCOMP_ROOT/var/lib/archive` or from a local :term:`SDS` archive [SDS archive] into a miniSEED file :file:`file.mseed` and sort by end time of the records: .. code-block:: sh scart -dsvE -t '[start-time]~[end-time]' [SDS archive] > [file.mseed] scart -dsvE -t '[start-time]~[end-time]' > file.mseed scart -dsvE -t '[start-time]~[end-time]' -n '[NET1],[NET2]' > file.mseed scart -dsvE -t '[start-time]~[end-time]' -n '[NET]' -c '(E,H)H(1,2,3)' > file.mseed scart -dsvE -t '[start-time]~[end-time]' -n '[N1.S1.L1.C1],[N2.S2.L2.C2]' > file.mseed scart -dsvE -t '[start-time]~[end-time]' --nslc list.file > file.mseed scart -dsvE -t --list list.file > file.mseed It is possible to achieve the same result of the dump mode using a combination of the input mode and the :ref:`scmssort` command, which allows to read the input data from any supported :ref:`global_recordstream`, not only an SDS archive: .. code-block:: sh scart -I [record-stream] --list list.file --stdout | scmssort -u -E -v > file.mseed .. note:: Sorting data is computational expensive but required for waveform playbacks. #. Push miniSEED data from file :file:`file.mseed` or standard input (stdin) into a local :term:`SDS` archive or a file. Additionally, you may check if the records of archived files are correctly ordered, filter by time and/or with NSLC list and print the output streams: .. code-block:: sh scmssort -u -E [file.mseed] > sorted.mseed scart -I sorted.mseed --with-filecheck [SDS archive] scart -I [file.mseed] -t '[start-time]~[end-time]' --print-streams --nslc list.file -o [out.mseed] cat sorted.mseed | scart -I - [SDS archive] cat sorted.mseed | scart [SDS archive] #. Collect data using the :ref:`global_recordstream` interface (e.g. FDSNWS server) and write to a miniSEED file or import it into a local :term:`SDS` archive. The data streams and the time spans can be defined in several ways. The data streams (:option:`--list`) can be automatically generated, e.g., by :ref:`scevtstreams`. .. code-block:: sh scart -I fdsnws://[server]:80 --list list.file [SDS archive] scart -I fdsnws://[server]:80 --list list.file -o file.mseed scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' --nslc list.file [SDS archive] scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[NET1],[NET2]' [SDS archive] scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[NET]' -c 'EH?' [SDS archive] scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[N1.S1.L1.C1],[N2.S2.L2.C2]' [SDS archive] #. Check all files of an SDS archive or other directory structure for miniSEED files with out-of-order records: .. code-block:: sh scart --check [archive] #. Print stream information from miniSEED files in archives or from :term:`RecordStream` without actually writing miniSEED data. In dump and import mode use the :option:`--test` if miniSEED data shall be read but not written. .. code-block:: sh scart --print-streams -I [miniSEED file] --test scart --print-streams -d -t [time span] --test [SDS archive] scart --print-streams --check [archive] The output looks like this: .. code-block:: sh # streamID start end records samples samplingRate GE.RGN..BHZ 2022-12-08T15:34:41.895Z 2022-12-08T15:52:19.145Z 58 21145 20.0 where the header and the text body are printed to stderr. Command-Line Options ==================== .. program:: scart :program:`scart [options] {archive-dir}` The last option has to be the archive directory when dump mode is enabled. When no archive directory is explicitly given, \$SEISCOMP_ROOT\/var\/lib\/archive or the current directory is used depending on whether \$SEISCOMP_ROOT has been set or not. The default operation mode is import. That means that a multiplexed MiniSEED file or another record source such as ArcLink is used to import records into a SDS structure. Verbosity --------- .. option:: -v, --verbose Verbose mode. .. option:: -h, --help Display a help message. Mode ---- .. option:: --check Check mode. Check all files in the given directory for erroneous miniSEED records. All sub\-directories are included. If no directory is given, the default SDS archive is scanned. Checks are only complete for files containing exactly one stream. More complete checks are made with scmssort. .. option:: -d, --dump Set export \(dump\) mode. Records are retrieved from an archive and written to standard output. .. option:: -I Import mode \(default\): Specify the recordstream URL to read the data from for archiving. When using any other recordstream than file, a stream list file is needed. Specifying \- implies file:\/\/\- \(stdin\). If no mode is explicitly specified, \-I file:\/\/\- is assumed. Processing ---------- .. option:: -c channels Channel filter to be applied to the data streams. Default for Dump: \"\(B\|E\|H\|M\|S\)\(D\|H\|L\|N\)\(E\|F\|N\|Z\|1\|2\|3\)\" Default for Import: \"\*\" .. option:: -E Dump mode: sort records according to their end time. Default: start time. .. option:: --files count Dump mode: Specify the number of file handles to cache. Default: 100. .. option:: -i Ignore records without data samples. .. option:: -l, --list file Import, dump mode: Use a stream list file with time windows instead of defined networks and channels \(\-n, \-c and \-t are ignored\). The list can be generated from events by scevtstreams. One line per stream. Line format: starttime;endtime;streamID The time format is the same as described in option '\-t'. Example: 2019\-07\-17 02:00:00;2019\-07\-17 02:10:00;GR.CLL..BH? .. option:: -m, --modify Dump mode: Modify the record time for real time playback. The first record time is NOW. The relative time of successive records to the first one are kept. .. option:: -n networks Import, dump mode: Data stream selection as a comma separated list \"stream1,stream2,streamX\" where each stream can be NET or NET.STA or NET.STA.LOC or NET.STA.LOC.CHA. If CHA is omitted, it defaults to the value of \-c option. Default: \"\*\" .. option:: --nslc file Import, dump mode: Stream list file to be used instead of defined networks and channels \(\-n and \-c are ignored\) for filtering the data by the given streams. Dump mode: Use in combination with \-t\! One line per stream, line format: NET.STA.LOC.CHA Example: GR.CLL..BH? .. option:: --rename rule Import, dump mode: Rename stream data according to the provided rule\(s\). A rule is \"[match\-stream:]rename\-stream\" and match\-stream is optional. match\-stream and rename\-stream are in the \"NET.STA.LOC.CHA\" format. match\-stream supports special charactes \"?\" \"\*\" \"\|\" \"\(\" \"\)\". rename\-stream supports the special character \"\-\" that can be used in place of NET, STA, LOC, CHA codes with the meaning of not renaming those. \"\-\" can also be used as the last character in CHA code. Multiple rules can be provided as a comma separated list or by providing multiple \-\-rename options. .. option:: -s, --sort Dump mode: Sort records by [start\-]time. To sort records by their end time use \-E. .. option:: --speed value Dump mode: Specify the speed to dump the records. A value of 0 means no delay otherwise speed is a multiplier of the real time difference between the records. When feeding the records directly into the replay pipe a value of 1 \(real time\) is recommended. .. option:: -t, --time-window timeWindow Import, dump mode: Specify the time window \(as one properly quoted string\) to dump records for. Times are UTC and separated by a tilde \"\~\". To dump one hour of waveform data between 2008\/01\/01 00:00:00 and 2008\/01\/01 01:00:00 use <\-t \"2008\-01\-01 00:00:00\~2008\-01\-01 01:00:00\">. Output ------ .. option:: -o, --output Dump, Import mode: Write data to given file instead of creating a SDS archive. Deactivates \-\-stdout. Deactivated by \-\-test. .. option:: --print-streams Print stream information only and exit. Works in import, dump and check mode. Output: NET.STA.LOC.CHA StartTime EndTime. .. option:: --stdout Import mode: Write to stdout instead of creating a SDS archive. Deactivated by \-\-test and \-\-output. .. option:: --test Test input only, deactivate all miniSEED output. This switch is useful for debugging and printing stream information with \-\-print\-streams. .. option:: --with-filecheck Import mode: Check all accessed files. Unsorted or unreadable files are reported to stderr. Checks are only complete for files containing exactly one stream. More complete checks are made with scmssort. .. option:: --with-filename Import mode: Print all accessed files to stderr after import.