.. index::
   single: FTMW
   single: FID
   single: chirps.csv
   single: clocks.csv
   single: markers.csv
   single: fidparams.csv
   single: processing.csv
   single: peakfind.csv
   single: overlays.csv
   single: Window Function
   single: Segment
   single: Frame

.. _data-storage-ftmw:

CP-FTMW Data
============

A CP-FTMW experiment writes the chirp, clock, and marker definitions
alongside the :doc:`general experiment files <experiment>`. The acquired
FIDs and their processing state live in a ``fid`` subfolder — numbered
``0.csv``, ``1.csv``, … plus ``fidparams.csv``, ``processing.csv``, and
``peakfind.csv`` — and overlays created for the experiment are saved
under an ``overlays`` subfolder and restored when the experiment is
reopened.

chirps.csv
----------

This file contains information about the CP-FTMW chirps. A single chirp
may be built from multiple segments (any of which may be empty), and each
segment has a starting frequency, an ending frequency, and a duration.
Example::

  Chirp;Segment;StartMHz;EndMHz;DurationUs;Alpha;Empty
  0;0;4895;1520;2;-1687.5;false

The first column (``Chirp``) is an index identifying the chirp, and the
second (``Segment``) identifies which segment of the chirp is being
described. In this experiment, a single chirp consists of one segment
starting at 4895 MHz and ending at 1520 MHz, with a duration of 2
microseconds. The ``Alpha`` column is the sweep rate in MHz/μs. If the
``Empty`` column is true, then the start and end values are ignored, and
the segment contains just 0 over the indicated duration.

An entire experiment may consist of many different chirps. The example
below shows an LO scan in which each of 20 identical chirps is recorded at
each LO tuning::

  Chirp;Segment;StartMHz;EndMHz;DurationUs;Alpha;Empty
  0;0;4895;1520;1;-3375;false
  1;0;4895;1520;1;-3375;false
  ...
  19;0;4895;1520;1;-3375;false

.. note::
   The frequencies and sweep rate in ``chirps.csv`` refer to AWG
   frequencies. The actual chirp range depends on the
   :ref:`RF Configuration <rf-configuration>`.

clocks.csv
----------

This file contains the configuration of the clocks (upconversion LO,
downconversion LO, etc.) as discussed on the
:ref:`RF Configuration <rf-configuration>` page. In a typical CP-FTMW
experiment, each clock is set to a single value. Example::

  Index;ClockType;FreqMHz;Operation;Factor;HwKey;OutputNum
  0;UpLO;11520;Multiply;2;Clock.virtual;0
  0;DownLO;40960;Multiply;8;Clock.virtual;1

In some cases (e.g., an :ref:`user_guide/experiment/acquisition_types:LO Scan`
or a :ref:`user_guide/experiment/acquisition_types:DR Scan`), one or more
of the clocks may be tuned to different values throughout the experiment.
The following is an excerpt from an LO scan in which the upconversion and
downconversion LOs were each stepped by 250 MHz::

  Index;ClockType;FreqMHz;Operation;Factor;HwKey;OutputNum
  0;DownLO;40960;Multiply;8;Clock.virtual;1
  0;UpLO;11520;Multiply;2;Clock.virtual;0
  0;DRClock;7000;Multiply;1;Clock.virtual;2
  1;DownLO;41210;Multiply;8;Clock.virtual;1
  1;UpLO;11770;Multiply;2;Clock.virtual;0
  1;DRClock;7000;Multiply;1;Clock.virtual;2
  2;DownLO;41460;Multiply;8;Clock.virtual;1
  2;UpLO;12020;Multiply;2;Clock.virtual;0
  2;DRClock;7000;Multiply;1;Clock.virtual;2

The ``Index`` column refers to each step of the experiment. ``ClockType``
identifies the particular clock role (UpLO, DownLO, AwgRef, DRClock,
DigRef, or ComRef). ``FreqMHz`` is the logical clock frequency in MHz.
``Operation`` (Multiply or Divide) and ``Factor`` account for any
frequency divider or multiplier on the clock output, and these values are
used to determine how to convert the logical frequencies into hardware
frequency. ``HwKey`` and ``OutputNum`` identify which piece of hardware
was used and which output (in the event that the clock has multiple
outputs).

markers.csv
-----------

.. index::
   single: Markers
   single: AWG markers

This file contains the AWG marker channel definitions for the experiment.
Marker channels are output signals generated by the AWG in synchrony with
the chirp waveform, used for purposes such as protecting the receiver
amplifier (Protection role), enabling an amplifier gate (Gate role),
triggering other instruments (Trigger role), or arbitrary user-defined
purposes (Custom role). The number of available marker channels depends on
the AWG model. For details on configuring marker channels in the user
interface, see :ref:`chirp-setup-markers`.

The columns are:

* ``Channel``: Zero-based marker channel index.
* ``Name``: User-defined label for the channel.
* ``Role``: One of ``Protection``, ``Gate``, ``Trigger``, or ``Custom``.
* ``TimingMode``: ``ChirpRelative`` (timing is repeated relative to each
  chirp start/end) or ``Absolute`` (timing is relative to the first chirp
  start and fires once per waveform).
* ``StartUs``: Start time in microseconds. For ``ChirpRelative`` mode,
  this is relative to the chirp start (negative values begin before the
  chirp starts). For ``Absolute`` mode, this is relative to the first
  chirp start.
* ``EndUs``: End time in microseconds. For ``ChirpRelative`` mode, this is
  relative to the chirp end (positive values extend past the chirp end).
* ``Enabled``: Whether the marker channel is active.

Example for a two-channel AWG with a protection pulse on channel 0 and an
amplifier gate on channel 1::

  Channel;Name;Role;TimingMode;StartUs;EndUs;Enabled
  0;Protection;Protection;ChirpRelative;-0.5;0.5;true
  1;Gate;Gate;ChirpRelative;-0.5;0.5;true

If no AWG is configured with marker support (``markerCount == 0``), this
file is not written.

fid/fidparams.csv
-----------------

This file contains the information needed to convert raw FID data into
numerical values, as well as the information needed to determine the
appropriate frequency values following a Fourier transform. Example::

  index;spacing;probefreq;vmult;shots;sideband;size
  0;2e-11;40960;0.000390625;100;LowerSideband;750000

In this example, 100 shots were recorded at a single clock configuration.
For LO scan experiments with multiple clock configurations, there will be
one row per configuration, and the ``index`` column identifies the
corresponding FID CSV file (e.g. ``0.csv``, ``1.csv``, etc.). The ``size``
column gives the number of points in the FID.

Blackchirp does not store the averaged digitizer voltage. Instead, it
stores *the sum of the raw digitizer readings*. To convert the FID values
to average voltage, multiply the numbers in the FID file by ``vmult`` and
divide by ``shots``. The ``vmult`` column contains the conversion between
digitization levels and voltage, while ``shots`` contains the number of
digitizer readings that have been summed.

For calculating the frequency axis of the FT, ``spacing`` gives the time
between samples in seconds, ``probefreq`` gives the downconversion LO
frequency in MHz, and ``sideband`` tells whether the FT frequency should
be added (UpperSideband or 0) or subtracted (LowerSideband or 1) from
``probefreq``.

.. _ftmw-fid-files:

fid/N.csv — FID files
---------------------

In an effort to balance plaintext readability, ease of integration with
other analysis software, and file size, the summed digitizer values are
written as **base-36 signed integers**. A simple example may begin with::

  fid0
  -7n
  -k
  10
  -p
  -21
  6j
  -8o
  -2v
  4c
  -2x
  -1s
  -11

The first row is a column label, and each subsequent row contains a single
FID point (e.g., -7n = -275). In some configurations, the FID file may
contain data from multiple frames, as shown in the example below which has
20 FIDs (only the first 9 points for each FID are shown)::

  fid0;fid1;fid2;fid3;fid4;fid5;fid6;fid7;fid8;fid9;fid10;fid11;fid12;fid13;fid14;fid15;fid16;fid17;fid18;fid19
  -33;-1u;-22;7z;-4r;-4r;36;-4t;-r;2m;-as;-bk;1g;-8j;-3u;-50;-73;-b1;1u;-5s
  -w;-5v;-4p;7u;-br;-2j;-2n;-7h;-3v;-8z;-5t;-89;-5p;-be;23;-4e;q;-2l;-4a;-ck
  4g;-5f;2t;h;-i9;-a3;-d1;-r;-n;-hg;6c;-4p;-k1;-99;-31;-2z;-6i;-a0;-3w;-bw
  32;-cr;i;-9q;-b4;-bi;-2w;4c;5s;-iv;72;-7m;-7a;-2j;-6s;-cj;-77;-hj;2z;-e5
  -22;-l9;-72;-af;-82;4;-3j;-6a;-8e;-9l;-59;-2g;3n;m;-ch;-el;-l;-f7;-e;-gc
  -4o;-fi;-2e;-c0;-bk;58;-8w;-dj;-bo;-2z;-7v;6d;-6p;-6f;-i2;-8p;-8l;-au;-49;-68
  -5o;-3p;-9;-bv;-cu;-3e;2v;-6c;-1y;1j;-6v;5b;-2x;-9f;-dl;-4y;-ex;-2f;-3o;-8
  -az;-33;-99;-4r;-ee;-9p;-8e;-2l;-dk;56;-fq;-3t;38;3a;-7f;-4a;-2b;3m;-e;-4t
  -bg;-82;-6s;-7r;-8k;-3o;-id;-2j;-i9;3f;-gw;-7c;-6b;-r;-57;-4v;-2o;-h;-3r;-20

fid/processing.csv
------------------

This file contains the default
:ref:`processing settings <user_guide/cp-ftmw:FID Processing Settings>`
associated with the FID data. An example is shown below::

  ObjKey;Value
  AutoscaleIgnoreMHz;0
  FidEndUs;15
  FidExpfUs;0
  FidRemoveDC;false
  FidStartUs;0
  FidWindowFunction;None
  FidZeroPadFactor;0
  FtUnits;6

The ``FtUnits`` value refers to a scaling factor of 10\ :sup:`N` (i.e., a
setting of 6 converts the FT from units of V to μV). The window functions
are shown below, where ``N`` is the number of samples and ``n`` ranges
from 0 to ``N-1``:

* None/Boxcar (0)

.. math::
   f(n) = 1

* Bartlett (1)

.. math::
   f(n) = 1-\left|\frac{2n}{N-1}-1\right|

* Blackman (2)

.. math::
   f(n) = 0.42 - 0.5\cos\frac{2\pi n}{N} + 0.08\cos\frac{4\pi n}{N}

* BlackmanHarris (3)

.. math::
   f(n) = 0.35875 - 0.48829\cos\frac{2\pi n}{N} + 0.14128\cos\frac{4\pi n}{N} - 0.1168\cos\frac{6\pi n}{N}

* Hamming (4)

.. math::
   f(n) = 0.54 - 0.46\cos\frac{2\pi n}{N}

* Hanning (5)

.. math::
   f(n) = 0.5 - 0.5\cos\frac{2\pi n}{N}

* KaiserBessel (6). I\ :sub:`0` = regular modified cylindrical Bessel
  function, β=14.0

.. math::
   f(n;\beta) = \frac{I_0\left(\beta\sqrt{1-\left[\frac{2x}{N-1}\right]^2}\right)}{I_0(\beta)},\quad x = n-\frac{N-1}{2}

fid/peakfind.csv
----------------

This file records the peak-finder search settings associated with the
experiment. It stores the values used by the
:ref:`Peak Find panel <user_guide/cp-ftmw:Peak Find>`, so an experiment
reopened later restores its own search parameters. An experiment with no
``peakfind.csv`` uses the application-wide defaults, which are updated
whenever the options are changed.

The file uses the standard ``ObjKey;Value`` metadata layout::

  ObjKey;Value
  PeakMaxFreqMHz;26000
  PeakMinFreqMHz;8000
  PeakNavHalfWidthMHz;2
  PeakPolyOrder;3
  PeakSnr;5
  PeakWindowSize;11

The keys correspond to the controls in the Peak Finding Options dialog:

- ``PeakMinFreqMHz``, ``PeakMaxFreqMHz`` — frequency range searched for
  peaks, in MHz.
- ``PeakSnr`` — signal-to-noise ratio threshold for detection.
- ``PeakWindowSize`` — Savitzky-Golay smoothing window size (odd).
- ``PeakPolyOrder`` — Savitzky-Golay polynomial order.
- ``PeakNavHalfWidthMHz`` — half-width, in MHz, of the window used when
  centering a plot on a selected peak.

overlays/overlays.csv
---------------------

For each :doc:`overlay </user_guide/overlays>` defined for the experiment,
the ``overlays`` subfolder stores a combination of CSV files that preserve
the overlay data and all of its configuration settings. This master file
lists all overlays associated with the
experiment. Each overlay is identified by its label, and the value
indicates the overlay type: 0 = Blackchirp Experiment, 1 = Catalog,
2 = Generic XY Data. The file also stores version information for
compatibility tracking. Example::

  ObjKey;Value
  BCBuildVersion;"508a6973c274ae9fcf24f0949ba70970b7c51d39"
  BCMajorVersion;2
  BCMinorVersion;0
  BCPatchVersion;0
  BCReleaseVersion;devel
  Exp17;0
  c047527_full;1

overlays/[label].* — per-overlay files
--------------------------------------

For each overlay listed in ``overlays.csv``, two additional files are
created:

**[label].settings.csv**
  Contains all configuration parameters for the overlay, including:

  - Source file path and overlay type-specific settings
  - Curve appearance properties (color, line style, thickness, visibility)
  - Data processing parameters (scaling, offsets, frequency filtering)
  - Plot assignment and display preferences
  - Version information for compatibility

  Example for a catalog overlay::

    ObjKey;Value
    catalogConvolutionEnabled;false
    catalogLineshapeType;Gaussian
    catalogLinewidthKHz;100
    catalogTransitionCount;87
    curve_color;#40963a
    curve_thickness;2
    enabled;true
    label;c047527_full
    sourceFile;/path/to/catalog.cat
    yScale;-25056766.9100865

**[label].data.csv**
  Contains the processed overlay data ready for display, with columns for
  frequency (X) and intensity (Y) values. The data format varies by
  overlay type:

  - **Catalog overlays**: Frequencies and intensities of peaks, along with
    other transition metadata (quantum numbers, etc.).
  - **Generic XY overlays**: The parsed and filtered XY data from the
    source file.
  - **Blackchirp Experiment overlays**: The Fourier transform data from
    the referenced experiment.

  The data files use standard CSV format with semicolon separators,
  maintaining consistency with other Blackchirp data files.