casa6

<casaxml xsi:schemaLocation="http://casa.nrao.edu/schema/casa.xsd file:///opt/casa/code/xmlcasa/xml/casa.xsd" xmlns="http://casa.nrao.edu/schema/psetTypes.html" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

​

<task type="function" name="partition" category="manipulation">

     <shortdescription>Task to produce Multi-MSs using parallelism</shortdescription>

​

<description>

    Partition is a task to create a Multi-MS out of an MS. General selection

    parameters are included, and one or all of the various data columns

    (DATA, LAG_DATA and/or FLOAT_DATA, and possibly MODEL_DATA and/or

    CORRECTED_DATA) can be selected.

    The partition task creates a Multi-MS in parallel, using the CASA MPI framework.

    The user should start CASA as follows in order to run it in parallel.

    1) Start CASA on a single node with 8 engines. The first engine will be used as the

       MPIClient, where the user will see the CASA prompt. All other engines will be used

       as MPIServers and will process the data in parallel.

           mpicasa -n 8 casa --nogui --log2term

           partition(.....)

    2) Running on a group of nodes in a cluster.

           mpicasa -hostfile user_hostfile casa ....

           partition(.....)

        where user_hostfile contains the names of the nodes and the number of engines to use 

        in each one of them. Example:

            pc001234a, slots=5

            pc001234b, slots=4

    If CASA is started without mpicasa, it is still possible to create an MMS, but

    the processing will be done in sequential.

​

    A multi-MS is structured to have a reference MS on the top directory and a

    sub-directory called SUBMSS, which contain each partitioned sub-MS. The

    reference MS contains links to the sub-tables of the first sub-MS. The other

    sub-MSs contain a copy of the sub-tables each. A multi-MS looks like this in disk.

​

    ls ngc5921.mms

    ANTENNA           FLAG_CMD     POLARIZATION  SPECTRAL_WINDOW  table.dat

    DATA_DESCRIPTION  HISTORY      PROCESSOR     STATE            table.info

    FEED              OBSERVATION  SORTED_TABLE  SUBMSS           WEATHER

    FIELD             POINTING     SOURCE        SYSCAL

​

    ls ngc5921.mms/SUBMSS/

    ngc5921.0000.ms/  ngc5921.0002.ms/  ngc5921.0004.ms/  ngc5921.0006.ms/

    ngc5921.0001.ms/  ngc5921.0003.ms/  ngc5921.0005.ms/

​

    Inside casapy, one can use the task listpartition to list the information

    from a multi-MS.

    When partition processes an MMS in parallel, each sub-MS is processed independently in an engine.

    The log messages of the engines are identified by the string MPIServer-#, where # gives the number

    of the engine running that process. When the task runs sequentially, it shows the MPIClient text

    in the origin of the log messages or does not show anything.

</description>

​

<input>

    <param type="path" name="vis" mustexist="true"><shortdescription>Name of input measurement set</shortdescription><description>Name of input measurement set</description>

        <value/>

    </param>

​

    <param type="string" name="outputvis"><shortdescription>Name of output measurement set</shortdescription><description>Name of output measurement set</description>

        <value/>

    </param>

        <param type="bool" name="createmms"><shortdescription>Should this create a multi-MS output</shortdescription><description>Should this create a multi-MS output</description>

            <value>True</value>

        </param>

        <param type="string" name="separationaxis" ignorecase="true" subparam="true"><shortdescription>Axis to do parallelization across(scan, spw, baseline, auto)</shortdescription><description>Axis to do parallelization across(scan, spw, baseline, auto)</description>

            <value>auto</value>

            <allowed kind="enum" ignorecase="true">

                <value>auto</value>

                <value>scan</value>

                <value>spw</value>

                <value>baseline</value>

            </allowed>

        </param>

        <param type="any" name="numsubms" subparam="true"><shortdescription>The number of SubMSs to create (auto or any number)</shortdescription><description>The number of SubMSs to create (auto or any number)</description>

            <type>string</type><type>int</type>

         <value type="string">auto</value>

        </param>

        <param type="bool" name="flagbackup" subparam="true"><shortdescription>Create a backup of the FLAG column in the MMS.</shortdescription><description>Create a backup of the FLAG column in the MMS.</description>

            <value>True</value>

        </param>

​

    <param type="string" name="datacolumn" ignorecase="True"><shortdescription>Which data column(s) to process.</shortdescription><description>Which data column(s) to process.</description>

        <value>all</value>

        <allowed kind="enum" ignorecase="True">

            <value>all</value>

            <value>data</value>

            <value>corrected</value>

            <value>model</value>

            <value>data,model,corrected</value>

            <value>float_data</value>

            <value>lag_data</value>

            <value>float_data,data</value>

            <value>lag_data,data</value>

        </allowed>

    </param>

​

    <param type="any" name="field"><shortdescription>Select field using ID(s) or name(s).</shortdescription><description>Select field using ID(s) or name(s).</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

​

    <param type="any" name="spw"><shortdescription>Select spectral window/channels.</shortdescription><description>Select spectral window/channels.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="scan"><shortdescription>Select data by scan numbers.</shortdescription><description>Select data by scan numbers.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

​

    <param type="any" name="antenna"><shortdescription>Select data based on antenna/baseline.</shortdescription><description>Select data based on antenna/baseline.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="correlation"><shortdescription>Correlation: '' ==&gt; all, correlation=&quot;XX,YY&quot;.</shortdescription><description>Correlation: '' ==&gt; all, correlation=&quot;XX,YY&quot;.</description>

        <type>string</type><type>stringVec</type>

        <value type="string"/>

    </param>

​

    <param type="any" name="timerange"><shortdescription>Select data by time range.</shortdescription><description>Select data by time range.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="intent"><shortdescription>Select data by scan intent.</shortdescription><description>Select data by scan intent.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="array"><shortdescription>Select (sub)array(s) by array ID number.</shortdescription><description>Select (sub)array(s) by array ID number.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="uvrange"><shortdescription>Select data by baseline length.</shortdescription><description>Select data by baseline length.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

    <param type="any" name="observation"><shortdescription>Select by observation ID(s).</shortdescription><description>Select by observation ID(s).</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param>

​

    <param type="any" name="feed"><shortdescription>Multi-feed numbers: Not yet implemented.</shortdescription><description>Multi-feed numbers: Not yet implemented.</description>

        <type>string</type><type>stringVec</type><type>int</type><type>intVec</type>

        <value type="string"/>

    </param> 

​

    <param type="bool" name="disableparallel" subparam="true" visibility="hidden"><shortdescription>Create a multi-MS in parallel.</shortdescription><description>Create a multi-MS in parallel.</description>

        <value>False</value>

    </param>        

    <param type="int" name="ddistart" subparam="true" visibility="hidden"><shortdescription>Do not change this parameter. For internal use only.</shortdescription><description>Do not change this parameter. For internal use only.</description>

        <value>-1</value>

    </param>

    <param type="string" name="taql" subparam="true" visibility="hidden"><shortdescription>Table query for nested selections</shortdescription><description>Table query for nested selections</description>

        <value type="string"/>

    </param>             

    <constraints>

      <when param="createmms">

        <equals type="bool" value="True">

          <default param="separationaxis"><value type="string">auto</value></default>

          <default param="numsubms"><value type="string">auto</value></default>

          <default param="flagbackup"><value type="bool">True</value></default>

​

          <default param="disableparallel"><value type="bool">False</value></default>

          <default param="ddistart"><value type="int">-1</value></default>

          <default param="taql"><value type="string"/></default>

        </equals>

​

        <equals type="bool" value="False">

          <default param="separationaxis"><value type="string">auto</value></default>

          <default param="numsubms"><value type="string">auto</value></default>

          <default param="flagbackup"><value type="bool">True</value></default>

​

          <default param="disableparallel"><value type="bool">False</value></default>

          <default param="ddistart"><value type="int">-1</value></default>

          <default param="taql"><value type="string"/></default>

        </equals>

      </when>

​

    </constraints>

​

</input>

​

<example>

​

​

----- Detailed description of keyword arguments -----

    vis -- Name of input visibility file

        default: none; example: vis='ngc5921.ms'

​

    outputvis -- Name of output visibility file

        default: none; example: outputvis='ngc5921.mms'

​

    createmms -- Create a multi-MS as the output.

        default: True

        If False, it will work like the split task and create a

        normal MS, split according to the given data selection parameters.

        Note that, when this parameter is set to False, a cluster

        will not be used.

​

        separationaxis -- Axis to do parallelization across. 

            default: 'auto'

            Options: 'scan', 'spw', 'baseline', 'auto'

​

            - The 'auto' option will partition per scan/spw to obtain optimal load balancing with the

             following criteria:

               1. Maximize the scan/spw/field distribution across sub-MSs

               2. Generate sub-MSs with similar size

​

            - The 'scan' or 'spw' axes will partition the MS into scan or spw. The individual sub-MSs may

            not be balanced with respect to the number of rows.

​

            - The 'baseline' axis is mostly useful for Single-Dish data. This axis will partition the MS

              based on the available baselines. If the user wants only auto-correlations, use the

              antenna selection such as antenna='\*&&&' together with this separation axis. Note that in

              if numsubms='auto', partition will try to create as many subMSs as the number of available

              servers in the cluster. If the user wants to have one subMS for each baseline, set the numsubms

              parameter to a number higher than the number of baselines to achieve this.        

        numsubms -- The number of sub-MSs to create.

            default: 'auto'

            Options: any integer number (example: numsubms=4)

               The default 'auto' is to partition using the number of available servers in the cluster.

               If the task is unable to determine the number of running servers, or the user did not start CASA

               using mpicasa, numsubms will use 8 as the default.

​

                Example: Launch CASA with 5 engines, where 4 of them will be used to create the MMS. The first

                    engine is used as the MPIClient.

                mpicasa -n 5 casa --nogui --log2term

                CASA> partition('uid__A1', outputvis='test.mms')

​

        flagbackup -- Make a backup of the FLAG column of the output MMS. When the

                      MMS is created, the .flagversions of the input MS are not transferred,

                      therefore it is necessary to re-create it for the new MMS. Note

                      that multiple backups from the input MS will not be preserved. This

                      will create a single backup of all the flags present in the input

                      MS at the time the MMS is created.

            default: True

​

    datacolumn -- Which data column to use when partitioning the MS.

        default='all'; example: datacolumn='data'

        Options: 'data', 'model', 'corrected', 'all',

                'float_data', 'lag_data', 'float_data,data', and

                'lag_data,data'.

            N.B.: 'all' = whichever of the above that are present.

​

---- Data selection parameters (see help par.selectdata for more detailed

    information)

​

    field -- Select field using field id(s) or field name(s).

             [run listobs to obtain the list iof d's or names]

        default: ''=all fields If field string is a non-negative

           integer, it is assumed to be a field index

           otherwise, it is assumed to be a field name

           field='0~2'; field ids 0,1,2

           field='0,4,5~7'; field ids 0,4,5,6,7

           field='3C286,3C295'; fields named 3C286 and 3C295

           field = '3,4C\*'; field id 3, all names starting with 4C

​

    spw -- Select spectral window/channels

        default: ''=all spectral windows and channels

           spw='0~2,4'; spectral windows 0,1,2,4 (all channels)

           spw='<2';  spectral windows less than 2 (i.e. 0,1)

           spw='0:5~61'; spw 0, channels 5 to 61

           spw='0,10,3:3~45'; spw 0,10 all channels, spw 3 - chans 3 to 45.

           spw='0~2:2~6'; spw 0,1,2 with channels 2 through 6 in each.

           spw = '\*:3~64'  channels 3 through 64 for all sp id's

                   spw = ' :3~64' will NOT work.

           spw = '\*:0;60~63'  channel 0 and channels 60 to 63 for all IFs 

                  ';' needed to separate different channel ranges in one spw

           spw='0:0~10;15~60'; spectral window 0 with channels 0-10,15-60

           spw='0:0~10,1:20~30,2:1;2;4'; spw 0, channels 0-10,

                    spw 1, channels 20-30, and spw 2, channels, 1, 2 and 4

​

    antenna -- Select data based on antenna/baseline

        default: '' (all)

            Non-negative integers are assumed to be antenna indices, and

            anything else is taken as an antenna name.

​

            Examples:

            antenna='5&6': baseline between antenna index 5 and index 6.

            antenna='VA05&VA06': baseline between VLA antenna 5 and 6.

            antenna='5&6;7&8': baselines 5-6 and 7-8

            antenna='5': all baselines with antenna 5

            antenna='5,6,10': all baselines including antennas 5, 6, or 10

            antenna='5,6,10&': all baselines with *only* antennas 5, 6, or

                                   10.  (cross-correlations only.  Use &&

                                   to include autocorrelations, and &&&

                                   to get only autocorrelations.)

            antenna='!ea03,ea12,ea17': all baselines except those that

                                       include EVLA antennas ea03, ea12, or

                                       ea17.

​

    timerange -- Select data based on time range:

        default = '' (all); examples,

           timerange = 'YYYY/MM/DD/hh:mm:ss~YYYY/MM/DD/hh:mm:ss'

           Note: if YYYY/MM/DD is missing date, timerange defaults to the

           first day in the dataset

           timerange='09:14:0~09:54:0' picks 40 min on first day

           timerange='25:00:00~27:30:00' picks 1 hr to 3 hr 30min

           on next day

           timerange='09:44:00' data within one integration of time

           timerange='>10:24:00' data after this time

​

    array -- (Sub)array number range

        default: ''=all

​

    uvrange -- Select data within uvrange (default units meters)

        default: ''=all; example:

            uvrange='0~1000klambda'; uvrange from 0-1000 kilo-lambda

            uvrange='>4klambda';uvranges greater than 4 kilo-lambda

            uvrange='0~1000km'; uvrange in kilometers

​

    scan -- Scan number range

        default: ''=all

​

    observation -- Select by observation ID(s)

        default: ''=all

​

​

------ EXAMPLES ------

​

1) Create a Multi-MS of some spws, partitioned per spw. The MS contains 16 spws.

    partition('uid001.ms', outpuvis='source.mms', spw='1,3~10', separationaxis='spw')

​

2) Create a Multi-MS but select only the first channels of all spws. Do not back up the FLAG

column.

    partition('uid0001.ms', outputvis='fechans.mms', spw='\*:1~10', flagbackup=False)

​

3) Create a Multi-MS using both separation axes.

    partition('uid0001.ms', outputvis='myuid.mms', createmms=True, separationaxis='auto')

​

4) Create a single-dish Multi-MS using the baseline axis only for the auto-correlations.

    partition('uid0001.ms', outputvis='myuid.mms', createmms=True, separationaxis='baseline', antenna='\*&&&')

​

​

</example> 

<returns>void</returns></task>

</casaxml>