Commits
Ville Suoranta authored 958747b827b Merge
1 1 | |
2 2 | |
3 3 | <casaxml xmlns="http://casa.nrao.edu/schema/psetTypes.html" |
4 4 | xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" |
5 5 | xsi:schemaLocation="http://casa.nrao.edu/schema/casa.xsd |
6 6 | file:///opt/casa/code/xmlcasa/xml/casa.xsd"> |
7 7 | |
8 8 | <task type="function" name="flagcmd" category="data editing"> |
9 9 | <shortdescription>Flagging task based on batches of flag-commands</shortdescription> |
10 10 | <description> |
11 - | The flagcmd task allows several batch-operations using flag commands. |
11 + | The flagcmd task will flag the visibility data or calibration |
12 + | table based on several batch-operations using flag commands. |
12 13 | |
13 - | Flag commands follow the mode and parameter names from the flagdata task |
14 - | (also explained below). The available modes are: manual, clip, shadow, quack, |
15 - | elevation, tfcrop, rflag and extend. The summary mode is not supported |
16 - | in this task. Use the flagdata task for that. |
14 + | Flag commands follow the mode and parameter names from the |
15 + | flagdata task. |
17 16 | |
18 - | The flagcmd task will flag data based on the commands input on inpmode : |
17 + | The flagcmd task will flag data based on the commands input on |
18 + | inpmode: |
19 19 | table = input from FLAG_CMD table in MS |
20 - | list = input from text file or list of strings given in inpfile |
20 + | list = input from text file or list of strings from inpfile |
21 21 | xml = input from Flag.xml in the MS given by vis |
22 22 | |
23 23 | Batch operations include : apply/unapply/list/plot/clear/extract |
24 24 | |
25 - | IMPORTANT: The FLAG_CMD sub-table is meant only for meta-data selections such as online flags. |
26 - | Using it to save other parameters (from modes such as clip, quack, shadow, etc) is |
27 - | possible but carries a risk that in future releases these parameters maybe renamed |
28 - | or changed their default values. Use it at your own risk! There will be no automatic |
29 - | way to rename any parameter that changes in the future. |
30 - | |
31 - | There is no way to guarantee that a command from the COMMAND column has been |
32 - | applied or not to the MS, even if the APPLIED column is set to True. If you |
33 - | use other ways to flag such as interactive flagging in plotms, the FLAG_CMD |
34 - | will not be updated! Use at your own risk. |
25 + | IMPORTANT: If you use other ways to flag such as interactive |
26 + | flagging in plotms, the FLAG_CMD will NOT be updated! |
35 27 | |
36 28 | NOTE on flagging calibration tables. |
37 - | ----------------------------------- |
38 - | |
39 - | It is possible to flag cal tables using this task, although we recommend using the flagdata |
40 - | task for this. |
41 - | |
42 - | When using this task to flag cal tables, only the 'apply' and 'list' actions are supported. |
43 - | Because cal tables do not have a FLAG_CMD sub-table, the default inpmode='table' can only |
44 - | be used if an MS is given in the 'inpfile' parameter so that flags from the MS are applied to |
45 - | the cal table. Otherwise, the flag commands must be given using inpmode='list', either from a |
46 - | file(s) or from a list of strings. See below for more information about these parameters. |
47 - | Data selection for calibration tables is limited to field, scan, antenna, time, spw and |
48 - | observation. If the calibration table was created before CASA 4.1, this task will create |
49 - | a dummy OBSERVATION column and OBSERVATION sub-table in the input calibration table to |
50 - | adapt it to the new cal table format. |
51 - | |
29 + | ----------------------------------- |
30 + | We recommend using the flagdata task for flagging cal tabels. When |
31 + | using flagcmd to flag cal tables, only the 'apply' and 'list' |
32 + | actions are supported. Because cal tables do not have a FLAG_CMD |
33 + | sub-table, the default inpmode='table' can only be used if an MS |
34 + | is given in the 'inpfile' parameter so that flags from the MS are |
35 + | applied to the cal table. Otherwise, the flag commands must be |
36 + | given using inpmode='list', either from a file(s) or from a list |
37 + | of strings. Data selection for calibration tables is limited to |
38 + | field, scan, antenna, time, spw and observation. |
39 + | |
52 40 | </description> |
53 41 | <input> |
54 42 | <param type="string" name="vis" mustexist="true"> |
55 - | <description>Name of MS file or calibration table to flag</description> |
43 + | <shortdescription>Name of MS file or calibration table to flag</shortdescription> |
44 + | <description>Name of input visibility file or calibration table. |
45 + | default: '' (none) |
46 + | |
47 + | example: vis='uid___A002_X2a5c2f_X54.ms' |
48 + | </description> |
56 49 | <value></value> |
57 50 | </param> |
58 51 | |
59 52 | <param type="string" name="inpmode"> |
60 - | <description>Input mode for flag commands(table/list/xml)</description> |
53 + | <shortdescription>Input mode for flag commands(table/list/xml)</shortdescription> |
54 + | <description>Input mode for flag commands(table/list/xml) |
55 + | options: 'table','list','xml' |
56 + | default: 'table' (the input commands from |
57 + | FLAG_CMD table of the MS) |
58 + | |
59 + | inpmode='xml' inputs online flags from Flag.xml |
60 + | file in the MS. This mode has become largely |
61 + | obsolete with the deprecation of the importevla |
62 + | task (see the flagcmd task pages in CASA Docs for |
63 + | more information). This mode will not work for |
64 + | ALMA MS or cal tables. |
65 + | |
66 + | NOTE: You can only apply the flags from a list or |
67 + | xml; you will not be able to unapply |
68 + | them. Transfer the flag commands to the FLAG_CMD |
69 + | table if you want to unapply the flags (see |
70 + | 'inpfile' description below). |
71 + | </description> |
61 72 | <value>table</value> |
62 73 | <allowed kind="enum"> |
63 74 | <value>table</value> |
64 75 | <value>list</value> |
65 76 | <value>xml</value> |
66 77 | </allowed> |
67 78 | </param> |
68 79 | |
69 80 | <param type="any" name="inpfile" subparam="true"> |
70 - | <description>Source of flag commands</description> |
81 + | <shortdescription>Source of flag commands</shortdescription> |
82 + | <description>Source of flag commands. Subparameter of |
83 + | inpmode='table/list'. |
84 + | Path to MS containing FLAG_CMD (table), or name |
85 + | of an ASCII file, list of files or a list of |
86 + | Python strings to apply to MS or cal table |
87 + | (list). |
88 + | options: [] with flag commands or [] with |
89 + | filenames or '' with a filename. (String values |
90 + | must contain quotes around them or the parser |
91 + | will not work.) |
92 + | default: '' (read from FLAG_CMD table in the MS |
93 + | specified via 'vis') |
94 + | |
95 + | Main use is to read flags from internal FLAG_CMD, |
96 + | but one use case is to read the flag commands |
97 + | from an MS given in inpfile and apply them to |
98 + | another MS or cal table given in vis. |
99 + | </description> |
71 100 | <any type="variant" limittypes="string stringArray"/> |
72 101 | <value type="string"></value> |
73 102 | </param> |
74 - | <param type="intArray" name="tablerows" subparam="true"> |
75 - | <description>Rows of inpfile to read</description> |
103 + | <param type="intArray" name="tablerows" |
104 + | subparam="true"> |
105 + | <shortdescription>Rows of inpfile to read</shortdescription> |
106 + | <description>List of rows of the FLAG_CMD table to read. Subparameter |
107 + | of inpmode='table/list'. |
108 + | default: [] (read all rows) |
109 + | |
110 + | example: [0,1,2,10] |
111 + | |
112 + | NOTE: currently only takes integer lists, not |
113 + | parseable strings with ranges. Use the Python |
114 + | range function to generate ranges, e.g. tablerows |
115 + | = range(0,30) + range(50,55) instead of |
116 + | '0~29,50~54' for now. |
117 + | </description> |
76 118 | <value type="vector"></value> |
77 119 | </param> |
78 120 | <param type="any" name="reason" subparam="true"> |
79 - | <description>Select by REASON types</description> |
121 + | <shortdescription>Select by REASON types</shortdescription> |
122 + | <description>Select flag commands based on REASON(s). Subparameter of |
123 + | inpmode. |
124 + | default: 'any' (all flags regardless of reason) |
125 + | |
126 + | Examples: |
127 + | reason='FOCUS_ERROR' |
128 + | reason=['FOCUS_ERROR','SUBREFLECTOR_ERROR'] |
129 + | |
130 + | If inpfile is a list of files, the reasons given |
131 + | in this parameter will apply to all the files. |
132 + | |
133 + | NOTE: what is within the string is literally |
134 + | matched, e.g. reason='' matches only blank |
135 + | reasons, and reason |
136 + | ='FOCUS_ERROR,SUBREFLECTOR_ERROR' matches this |
137 + | compound reason string only |
138 + | </description> |
80 139 | <any type="variant" limittypes="string stringArray"/> |
81 140 | <value type="string">any</value> |
82 141 | </param> |
83 142 | <param type="bool" name="useapplied" subparam="true"> |
84 - | <description>Select commands whose rows have APPLIED column set to True</description> |
143 + | <shortdescription>Select commands whose rows have APPLIED column set to True</shortdescription> |
144 + | <description>Select commands whose rows have APPLIED column set to |
145 + | True. Subparameter of inpmode='table'. |
146 + | options: True,False |
147 + | default: False |
148 + | |
149 + | If useapplied=True it will read in both applied |
150 + | and unapplied flags. |
151 + | |
152 + | IMPORTANT: The APPLIED column is set to True |
153 + | after a flag command is applied to the MS. In |
154 + | order to re-apply the same flag command, this |
155 + | parameter should be set to True. |
156 + | </description> |
85 157 | <value>False</value> |
86 158 | </param> |
87 159 | |
88 160 | <param type="double" name="tbuff" subparam="true"> |
89 - | <description>Time buffer (sec) to pad flags</description> |
161 + | <shortdescription>Time buffer (sec) to pad flags</shortdescription> |
162 + | <description>Time buffer (sec) to pad flags. Subparameter of |
163 + | inpmode='xml'. |
164 + | default: 0.0 |
165 + | </description> |
90 166 | <value>0.0</value> |
91 167 | </param> |
92 168 | |
93 169 | <param type="string" name="ants" subparam="true"> |
94 - | <description>Allowed flag antenna names to select by</description> |
170 + | <shortdescription>Allowed flag antenna names to select by</shortdescription> |
171 + | <description>Allowed flag antenna names to select by. Subparameter of |
172 + | inpmode='xml'. |
173 + | </description> |
95 174 | <value></value> |
96 175 | </param> |
97 176 | |
98 177 | <param type="string" name="action"> |
99 - | <description>Action to perform in MS and/or in inpfile (apply/unapply/list/plot/clear/extract)</description> |
178 + | <shortdescription>Action to perform in MS and/or in inpfile (apply/unapply/list/plot/clear/extract)</shortdescription> |
179 + | <description>Action to perform in MS and/or in inpfile |
180 + | options: apply/unapply/list/plot/clear/extract |
181 + | default: 'apply' |
182 + | |
183 + | Examples: |
184 + | -- action='apply': This operation will apply |
185 + | the commands chosen by inpmode. If |
186 + | inpmode='table' and inpfile='' then the |
187 + | APPLIED column in FLAG_CMD will be set to |
188 + | True. |
189 + | -- action='unapply': unapply flags in MS. (Not |
190 + | available for cal tables). This operation will |
191 + | unapply the commands chosen by inpmode='table' |
192 + | ONLY. After unapplying the commands, the task |
193 + | will update the APPLIED column to False. |
194 + | -- action='list': list and/or save flag |
195 + | commands. This operation will list the |
196 + | commands chosen by inpmode on the screen and |
197 + | save them to the MS or to a file without |
198 + | applying. It will save the commands to outfile |
199 + | if the parameter savepars is set to True. If |
200 + | outfile is None, it will save the commands to |
201 + | the MS given in 'vis'. |
202 + | -- action='plot': plot flags (ant |
203 + | vs. time). (Not available for cal |
204 + | tables). This operation will plot the flags |
205 + | chosen by inpmode to a matplotlib gui or to a |
206 + | file. These will be sorted by antenna |
207 + | vs. time. Most useful for showing the online |
208 + | flags. |
209 + | -- action='clear': clear flags from FLAG_CMD |
210 + | in the MS. (Not available for cal tables) This |
211 + | operation will delete the selected flag rows |
212 + | from the internal FLAG_CMD table of the MS. |
213 + | -- action='extract': extract internal flag |
214 + | dictionary. (Not available for cal tables) |
215 + | This option will return the internal flagging |
216 + | dictionary to python. There is no extant |
217 + | description of the format of this dictionary, |
218 + | as it is an internal device used by the |
219 + | flagcmd task. This action is provided for the |
220 + | convenience of advanced users. |
221 + | |
222 + | WARNING: choosing this action='clear' will |
223 + | disregard anything you set in inpmode and will |
224 + | always work on the FLAG_CMD table in vis. This can |
225 + | be used to totally delete rows from the FLAG_CMD |
226 + | table, when setting clearall=True. |
227 + | </description> |
100 228 | <value>apply</value> |
101 229 | <allowed kind="enum"> |
102 230 | <value>apply</value> |
103 231 | <value>unapply</value> |
104 232 | <value>list</value> |
105 233 | <value>plot</value> |
106 234 | <value>clear</value> |
107 235 | <value>extract</value> |
108 236 | </allowed> |
109 237 | </param> |
110 238 | |
111 239 | <param type="bool" name="flagbackup" subparam="true"> |
112 - | <description>Automatically backup the FLAG column before execution</description> |
240 + | <shortdescription>Automatically backup the FLAG column before execution</shortdescription> |
241 + | <description>Automatically backup the FLAG column before |
242 + | execution. Subparameter of action='apply/unapply'. |
243 + | options: True,False |
244 + | default: True |
245 + | </description> |
113 246 | <value>True</value> |
114 247 | </param> |
115 248 | |
116 249 | <param type="bool" name="clearall" subparam="true"> |
117 - | <description>Delete all rows from FLAG_CMD</description> |
250 + | <shortdescription>Delete all rows from FLAG_CMD</shortdescription> |
251 + | <description>Delete all rows from FLAG_CMD. Subparameter of |
252 + | action='clear'. |
253 + | default: False (will not clear) |
254 + | </description> |
118 255 | <value>False</value> |
119 256 | </param> |
120 257 | |
121 258 | <param type="intArray" name="rowlist" subparam="true"> |
122 - | <description>FLAG_CMD rows to clear</description> |
259 + | <shortdescription>FLAG_CMD rows to clear</shortdescription> |
260 + | <description>FLAG_CMD rows to clear. Subparameter of action='clear'. |
261 + | default: [] (all flags in table) |
262 + | |
263 + | example: [0,1,2,10] |
264 + | |
265 + | WARNING: this can be dangerous, and you must set |
266 + | clearall=True to use this!!! This will delete |
267 + | the specified rows from the internal FLAG_CMD |
268 + | table for vis regardless of what mode is set to |
269 + | (useful for when you import from xml or file), |
270 + | and decide to redo it). This action will NOT |
271 + | unapply the commands. |
272 + | |
273 + | NOTE: currently only takes integer lists, not |
274 + | parseable strings with ranges. Use the Python |
275 + | range function to generate ranges, e.g. rowlist = |
276 + | range(0,30) + range(50,55) instead of |
277 + | '0~29,50~54' for now. |
278 + | </description> |
123 279 | <value type="vector"></value> |
124 280 | </param> |
125 281 | |
126 282 | <param type="string" name="plotfile" subparam="true"> |
127 - | <description>Name of output file to save plot</description> |
283 + | <shortdescription>Name of output file to save plot</shortdescription> |
284 + | <description>Name of output file to save plot |
285 + | default: '' (plot to matplotlib window) |
286 + | |
287 + | WARNING: will only reliably plot individual flags |
288 + | per antenna and timerange (e.g. direct from xml) |
289 + | </description> |
128 290 | <value></value> |
129 291 | </param> |
130 292 | |
131 293 | <param type="bool" name="savepars"> |
132 - | <description>Save flag commands to the MS or to a file</description> |
294 + | <shortdescription>Save flag commands to the MS or file</shortdescription> |
295 + | <description>Save the flag commands to the FLAG_CMD table of the MS or |
296 + | to an output text file. |
297 + | options: True/False |
298 + | default: False |
299 + | </description> |
133 300 | <value>False</value> |
134 301 | </param> |
135 302 | <param type="string" name="outfile" subparam="true"> |
136 - | <description>Name of output file to save commands</description> |
303 + | <shortdescription>Name of output file to save commands</shortdescription> |
304 + | <description>Name of output file to save commands. Subparameter of |
305 + | savepars=True. |
306 + | default: ' '; it will save the commands in the |
307 + | FLAG_CMD table of the MS. |
308 + | |
309 + | example: outfile='flags.txt' will save the |
310 + | parameters in a text file. |
311 + | </description> |
137 312 | <value></value> |
138 313 | </param> |
139 314 | <param type="bool" name="overwrite" subparam="true"> |
140 - | <description>Overwrite an existing file to save the flag commands</description> |
315 + | <shortdescription>Overwrite an existing file to save the flag commands</shortdescription> |
316 + | <description>Overwrite an existing file given in 'outfile' to save the |
317 + | flag commands. Subparameter of savepars=True. |
318 + | options: True/False |
319 + | default: True; it will remove the existing file |
320 + | given in 'outfile' and save the current flag |
321 + | commands to a new file with the same name. When |
322 + | set to False, the task will exit with an error |
323 + | message if the file exist. |
324 + | </description> |
141 325 | <value>True</value> |
142 326 | </param> |
143 327 | |
144 328 | |
145 329 | <constraints> |
146 330 | <when param="inpmode"> |
147 331 | <equals value="table"> |
148 332 | <default param="inpfile"><value type="string"></value></default> |
149 333 | <default param="tablerows"><value type="vector"></value></default> |
150 334 | <default param="reason"><value type="string">any</value></default> |
185 369 | <default param="overwrite"><value type="bool">True</value></default> |
186 370 | </equals> |
187 371 | </when> |
188 372 | |
189 373 | </constraints> |
190 374 | </input> |
191 375 | <returns type="void"/> |
192 376 | |
193 377 | <example> |
194 378 | |
195 - | Keyword arguments: |
196 - | |
197 - | vis -- Name of input visibility file or calibration table. |
198 - | default: '' (none) |
199 - | example1: vis='uid___A002_X2a5c2f_X54.ms' or |
200 - | example2: vis='cal-X54.B1' |
201 - | |
202 - | -- INPUT of flag commands -- |
203 - | |
204 - | inpmode -- Input mode for flag commands. |
205 - | options: 'table','list','xml' |
206 - | default: 'table' |
207 - | |
208 - | inpmode "table" -- input commands from FLAG_CMD table of MS. |
209 - | |
210 - | inpfile -- path to MS containing FLAG_CMD |
211 - | default: '' (read from FLAG_CMD table in the MS specified via 'vis') |
212 - | |
213 - | Main use is to read flags from internal FLAG_CMD, |
214 - | but by setting inpfile to a different MS you can |
215 - | use this to copy the flags from one MS to another. |
216 - | |
217 - | One use case is to read the flag commands from an MS given in |
218 - | inpfile and apply them to a cal table given in vis. Example: |
219 - | |
220 - | flagcmd(vis='cal-X54.B1', inpmode='table', inpfile='uid___A002_X2a5c2f_X54.ms', action='apply') |
221 - | |
222 - | tablerows -- list of rows of the FLAG_CMD table to read |
223 - | default: [] (read all rows) |
224 - | example: [0,1,2,10] |
225 - | |
226 - | NOTE: currently only takes integer lists, not |
227 - | parseable strings with ranges. Use the Python |
228 - | range function to generate ranges, e.g. |
229 - | tablerows = range(0,30) + range(50,55) |
230 - | instead of '0~29,50~54' for now. |
231 - | |
232 - | reason -- select flag commands based on REASON(s) |
233 - | default: 'any' (all flags regardless of reason) |
234 - | can be a string, or list of strings |
235 - | example: reason='FOCUS_ERROR' |
236 - | reason=['FOCUS_ERROR','SUBREFLECTOR_ERROR'] |
237 - | |
238 - | NOTE: what is within the string is literally |
239 - | mateched, e.g. reason='' matches only blank reasons, |
240 - | and reason = 'FOCUS_ERROR,SUBREFLECTOR_ERROR' |
241 - | matches this compound reason string only |
242 - | |
243 - | useapplied -- select the flag commands of rows that have column APPLIED=True |
244 - | options: True,False |
245 - | default: False |
246 - | |
247 - | If useapplied=True it will read in both applied and |
248 - | unapplied flags. |
249 - | |
250 - | IMPORTANT: The APPLIED column is set to True after a flag command is applied |
251 - | to the MS. In order to re-apply the same flag command, this |
252 - | parameter should be set to True. |
253 - | |
254 - | |
255 - | inpmode "list" -- input commands from an ASCII file, a list of files or via a list of strings. |
256 - | NOTE: You can only apply the flags from a list; you will not be able |
257 - | to unapply them. Transfer the flag commands to the FLAG_CMD table |
258 - | if you want to unapply the flags. For this, see action='list' below. |
259 - | |
260 - | inpfile -- name of an ASCII file, list of files or a list of Python strings to apply to |
261 - | MS or cal table. |
262 - | default: '' |
263 - | options: [] with flag commands or |
264 - | [] with filenames or |
265 - | '' with a filename. |
266 - | |
267 - | IMPORTANT: string values must contain quotes around them or the parser |
268 - | will not work. The parser evaluates the commands in the list |
269 - | and considers only existing Python types. |
270 - | |
271 - | example1: the following commands can be saved to a file or group of files |
272 - | and given to the task (e.g. save it to flags.txt). |
273 - | |
274 - | scan='1~3' mode='manual' |
275 - | mode='clip' clipminmax=[0,2] correlation='ABS_XX' clipoutside=False |
276 - | spw='9' mode='tfcrop' correlation='ABS_YY' ntime=51.0 |
277 - | mode='extend' extendpols=True |
278 - | |
279 - | flagcmd(vis, inpmode='list',inpfile='flags.txt') or |
280 - | flagcmd(vis, inpmode='list', inpfile=['onlineflags.txt','flags.txt']) |
281 - | |
282 - | |
283 - | example2: the same commands can be written in a Python list of strings and given |
284 - | to the task. |
285 - | cmd=["scan='1~3' mode='manual'", |
286 - | "mode='clip' clipminmax=[0,2] correlation='ABS_XX' clipoutside=False", |
287 - | "spw='9' mode='tfcrop' correlation='ABS_YY' ntime=51.0", |
288 - | "mode='extend' extendpols=True"] |
289 - | |
290 - | flagcmd(vis, inpmode='list',inpfile=cmd) |
291 - | |
292 - | |
293 - | reason -- select flag commands to apply, based on REASON(s) |
294 - | default: 'any' (all flags regardless of reason) |
295 - | can be a string, or list of strings |
296 - | example: reason='FOCUS_ERROR' |
297 - | reason=['FOCUS_ERROR','SUBREFLECTOR_ERROR'] |
298 - | |
299 - | If inpfile is a list of files, the reasons given in this |
300 - | parameter will apply to all the files. |
301 - | |
302 - | NOTE: what is within the string is literally |
303 - | mateched, e.g. reason='' matches only blank reasons, |
304 - | and reason = 'FOCUS_ERROR,SUBREFLECTOR_ERROR' |
305 - | matches this compound reason string only |
306 - | |
307 - | |
308 - | inpmode "xml" -- input online flags from Flag.xml file in the MS. This mode |
309 - | is not available for cal tables. This works only for MSs |
310 - | imported using importevla. It will not work for ALMA MSs. |
311 - | NOTE: You can only apply the flags from a XML file; you will not be able |
312 - | to unapply them. Transfer the flag commands to the FLAG_CMD table |
313 - | if you want to unapply the flags. For this, see action='list' below. |
314 - | |
315 - | tbuff -- (float) time padding buffer (seconds, default=1.0) |
316 - | |
317 - | ants -- select flags based on antenna, |
318 - | e.g. antenna='ea01' |
319 - | default: '' (all flags regardless of antenna) |
320 - | |
321 - | reason -- select flag commands based on REASON(s), |
322 - | default: 'Any' (all flags regardless of reason) |
323 - | can be a string, or list of strings |
324 - | example: reason='FOCUS_ERROR' |
325 - | reason=['FOCUS_ERROR','SUBREFLECTOR_ERROR'] |
326 - | |
327 - | NOTE: what is within the string is literally |
328 - | mateched, e.g. reason='' matches only blank reasons, |
329 - | and reason = 'FOCUS_ERROR,SUBREFLECTOR_ERROR' |
330 - | matches this compound reason string only |
331 - | |
332 - | --ACTIONS-- |
333 - | |
334 - | action -- operation to perform on MS and/or in flag commands from inpfile. |
335 - | options: 'apply','clear','list','plot','unapply','extract' |
336 - | default: 'apply' |
337 - | |
338 - | action "apply" -- apply flags to MS or cal table. |
339 - | |
340 - | This operation will apply the commands chosen by inpmode. |
341 - | If inpmode='table' it will set the APPLIED column to True. |
342 - | |
343 - | flagbackup -- Automatically backup MS/cal table FLAG column before applying. |
344 - | options: True,False |
345 - | default: True |
346 - | |
347 - | |
348 - | action "unapply" -- unapply flags in MS. (Not available for cal tables). |
349 - | |
350 - | This operation will unapply the commands chosen by inpmode='table' ONLY. |
351 - | After unapplying the commands, the task will update the APPLIED column to False. |
352 - | |
353 - | flagbackup -- Automatically backup MS FLAG column before unapplying? |
354 - | options: True,False |
355 - | default: True |
356 - | |
357 - | |
358 - | action "list" -- list and/or save flag commands. |
359 - | |
360 - | This operation will list the commands chosen by inpmode on the screen |
361 - | and save them to the MS or to a file without applying. It will save the commands |
362 - | to outfile if the parameter savepars is set to True. If outfile is None, it |
363 - | will save the commands to the MS given in 'vis'. |
364 - | |
365 - | |
366 - | action "plot" -- plot flags (ant vs. time). (Not available for cal tables) |
367 - | |
368 - | This operation will plot the flags chosen by inpmode to a |
369 - | matplotlib gui or to a file. These will be sorted by |
370 - | antenna vs. time. Most useful for showing the online |
371 - | flags. |
372 - | |
373 - | plotfile -- output plot file |
374 - | default: '' (plot to matplotlib window) |
375 - | |
376 - | WARNING: will only reliably plot individual flags |
377 - | per antenna and timerange (e.g. direct from xml) |
378 - | |
379 - | action "clear" -- clear flags from FLAG_CMD in MS. (Not available for cal tables) |
380 - | |
381 - | This operation will delete the selected flag rows from |
382 - | the internal FLAG_CMD table of the MS. |
383 - | |
384 - | NOTE: choosing this option will disregard anything you |
385 - | set in inpmode and will always work on the FLAG_CMD table |
386 - | in vis. |
387 - | |
388 - | clearall -- really clear all flags? |
389 - | default: False (will not clear) |
390 - | |
391 - | rowlist -- list of FLAG_CMD rows to clear |
392 - | default: [] (all flags in table) |
393 - | example: [0,1,2,10] |
394 - | |
395 - | NOTE: currently only takes integer lists, not |
396 - | parseable strings with ranges. Use the Python |
397 - | range function to generate ranges, e.g. |
398 - | rowlist = range(0,30) + range(50,55) |
399 - | instead of '0~29,50~54' for now. |
400 - | |
401 - | WARNING: this can be dangerous, and you must set clearall=True |
402 - | to use this!!! This will delete the specified rows from the |
403 - | internal FLAG_CMD table for vis regardless of what mode is set |
404 - | to (useful for when you import from xml or file), and decide to |
405 - | redo it). This action will NOT unapply the commands. |
406 - | |
407 - | |
408 - | action "extract" -- extract internal flag dictionary. (Not available for cal tables) |
409 - | |
410 - | This option will return the internal flagging dictionary to |
411 - | python. |
412 - | |
413 - | |
414 - | savepars -- Save the flag commands to the FLAG_CMD table of the MS or to an output text file. |
415 - | default: False |
416 - | options: True/False |
417 - | |
418 - | outfile -- Name of output file to save the flag commands. |
419 - | default: ' '; it will save the commands in the FLAG_CMD table of the MS. |
420 - | example: outfile='flags.txt' will save the parameters in a text file. |
421 - | |
422 - | overwrite -- Overwrite the existing file given in 'outfile'. |
423 - | options: True/False |
424 - | default: True; it will remove the existing file given in 'outfile' and save the current flag |
425 - | commands to a new file with the same name. When set to False, the task will exit with |
426 - | an error message if the file exist. |
427 - | |
428 - | =========================================================================== |
429 - | |
430 - | -- Internal FLAG_CMD input 'inpmode' useage -- |
431 - | |
432 - | (For inpmode='table') |
433 - | |
434 - | * It is a good idea to use action='list' first to see what is there |
435 - | before doing anything else, e.g. |
436 - | |
437 - | inpmode = 'table' |
438 - | action = 'list' |
439 - | |
440 - | * To apply the flags stored in the FLAG_CMD table in the MS, |
441 - | simply set inpmode='table' and action='apply'. This is the default setup |
442 - | of flagcmd. Note that when a flag command is applied, the corresponding APPLIED |
443 - | column cell will be updated to True. |
444 - | |
445 - | inpmode = 'table' |
446 - | action = 'apply' |
447 - | useapplied = False |
448 - | |
449 - | * To re-apply the flags stored in the FLAG_CMD table in the MS, |
450 - | inpmode='table', action='apply' and useapplied=True. |
451 - | |
452 - | inpmode = 'table' |
453 - | action = 'apply' |
454 - | useapplied = True |
455 - | |
456 - | |
457 - | * To merely save to FLAG_CMD but not apply, then |
458 - | |
459 - | inpmode = 'table' |
460 - | inpfile = 'other.ms' |
461 - | action = 'list' |
462 - | |
463 - | * To save commands from a file into the MS without applying. |
464 - | |
465 - | inpmode = 'list' |
466 - | inpfile = 'flags.txt' |
467 - | action = 'list' |
468 - | |
469 - | If you need to select only certain rows from the FLAG_CMD table, |
470 - | use the tablerows parameter to control this. Currently this must |
471 - | be a list of individual row numbers (0-based), e.g. |
472 - | |
473 - | tablerows = [0,1,2,3,10,11] |
474 - | |
475 - | or |
476 - | |
477 - | tablerows = range(29) |
478 - | |
479 - | NOTE: the useapplied=True/False tag is important if you are |
480 - | going to (re)apply flags marked as APPLIED True in FLAG_CMD. |
481 - | It is common to have a "failed" flagging operation mark the flags |
482 - | as already applied and then they don't show up when you re-run |
483 - | (e.g. in 'list'). Set useapplied=True so that it will use these |
484 - | anyway. |
485 - | |
486 - | * To apply the flag commands from an MS to a calibration table. |
487 - | |
488 - | vis = 'mycaltable' |
489 - | inpmode = 'table' |
490 - | inpfile = 'myMS.ms' |
491 - | action = 'apply' |
492 - | |
493 - | |
494 - | -- Online flag input inpmode useage -- |
495 - | |
496 - | (For inpmode='xml') |
497 - | |
498 - | * To list the online flags stored in the Flag.xml file in the |
499 - | MS, simply set: |
500 - | |
501 - | inpmode = 'xml' |
502 - | action = 'list' |
503 - | savepars = False |
504 - | |
505 - | * It is then straightforward to save these to FLAG_CMD |
506 - | |
507 - | inpmode = 'xml' |
508 - | action = 'list' |
509 - | savepars = True |
510 - | |
511 - | * To directly apply the online flags stored in the Flag.xml file in the |
512 - | MS, set inpmode='xml' and desired buffer, e.g. |
513 - | |
514 - | inpmode = 'xml' |
515 - | tbuff = 1.0 # pad flag times by 1 sec |
516 - | action = 'apply' |
517 - | set savepars to save or not the commands in the MS |
518 - | |
519 - | * You can also specify a set of reasons (a comma separated list) |
520 - | for flags to apply, e.g. |
521 - | |
522 - | reason = 'FOCUS_ERROR,SUBREFLECTOR_ERROR' # select these flags |
523 - | reason = 'ANTENNA_NOT_ON_SOURCE' |
524 - | |
525 - | NOTE: The online flag time buffer tbuff is specified in |
526 - | seconds, but in fact should be keyed to the intrinsic online |
527 - | integration time. This is particularly true for EVLA data, |
528 - | were a tbuff value of 0.5x to 1.5x the integration time is |
529 - | needed (currently you should use 1.5x for data taken in |
530 - | early 2011 or before). |
531 - | |
532 - | Because the Flag.xml is copied to the MS by importevla, |
533 - | you can re-apply the online flags with an increased tbuff |
534 - | simply by running with inpmode='xml' and optype='apply', e.g. |
535 - | |
536 - | inpmode = 'xml' |
537 - | tbuff = 15.0 # pad flag times by 15 sec for 10sec integrations |
538 - | optype = 'apply' |
539 - | |
540 - | if you originally used a smaller value (e.g. 1.0) by mistake |
541 - | or you want to try longer values. Note these will be added to |
542 - | the FLAG_CMD table which you would have to clean up manually |
543 - | if you care about this. |
544 - | |
545 - | |
546 - | -- Flag command useage -- |
547 - | |
548 - | (For inpmode='list') |
549 - | |
550 - | * For example, a series of commands might be: |
551 - | |
552 - | antenna='ea01' timerange='00:00:00~01:00:00' |
553 - | antenna='ea11' timerange='00:00:00~03:00:00' spw='0~4' |
554 - | mode='clip' clipminmax=[0,5] correlation='ABS_ALL' |
555 - | mode='quack' quackmode='end' quackinterval=1.0 |
556 - | mode='shadow' |
557 - | |
558 - | Any other mode can also use selection (see the help of flagdata): |
559 - | |
560 - | mode='shadow' antenna='ea01,ea02,ea03' |
561 - | mode='quack' quackmode='end' quackinterval=1.0 antenna='ea22' |
562 - | |
563 - | These commands can be saved in an ASCII file, e.g. "myflags.txt" |
564 - | and input using inpmode='list', e.g. |
565 - | |
566 - | flagcmd(vis='myvis.ms',inpmode='list',inpfile='myflags.txt') |
567 - | |
568 - | or input from the interface |
569 - | |
570 - | flagcmd(vis='myvis.ms',inpmode='list', |
571 - | inpfile=["mode='shadow'", |
572 - | "mode='clip' clipminmax=[0,5] correlation='ABS_ALL'", |
573 - | "mode='quack' quackmode='end' quackinterval=1.0", |
574 - | "antenna='ea01' timerange='00:00:00~01:00:00'", |
575 - | "antenna='ea11' timerange='00:00:00~03:00:00' spw='0~4'"]) |
379 + | For more information, see the task pages of flagcmd in CASA Docs: |
576 380 | |
577 - | |
578 - | -- ACTIONs -- |
579 - | |
580 - | The action parameter controls what flagcmd will actually do with the |
581 - | flag commands: |
582 - | |
583 - | * action = 'apply' |
584 - | |
585 - | This will apply the selected commands to the data. |
586 - | |
587 - | If inpmode='table' and inpfile='' then the APPLIED column in FLAG_CMD |
588 - | will be set to True. |
589 - | |
590 - | * action = 'unapply' |
591 - | |
592 - | This will unapply any commands on the selected data that |
593 - | come from the FLAG_CMD table. |
594 - | |
595 - | * action = 'list' |
596 - | |
597 - | List what is selected and or save in the MS or in a file. It is wise to do |
598 - | this first before doing any other action. It will list the output in the logger |
599 - | and save them to the FLAG_CMD table of the MS when savepars=True and outfile = '' or |
600 - | to a file if outfile is non-blank. |
601 - | |
602 - | * action = 'plot' |
603 - | |
604 - | Will pop up a little matplotlib GUI (if outfile='') or plot to a |
605 - | file. Currently only gives an antenna vs time plot, mostly useful |
606 - | for looking at the online flags. |
607 - | |
608 - | * action = 'clear' |
609 - | |
610 - | DANGER! This can be used to totally delete rows from the FLAG_CMD |
611 - | table. It ignores what inpmode is pointing to and always works |
612 - | on FLAG_CMD. Use at your own peril but sometimes you need to just |
613 - | blow that table away, e.g. |
614 - | |
615 - | vis = msfile |
616 - | optype = 'clear' |
617 - | rowlist = [] # all rows |
618 - | clearall = True # disarm the safety |
619 - | |
620 - | Note you have to explicitly set clearall=True to arm the deletion |
621 - | (a minimal precaution). |
622 - | |
623 - | * action = 'extract' |
624 - | |
625 - | This option will return the internal flagging dictionary to |
626 - | python. This will allow a power-user to manipulate these |
627 - | commands directly (e.g. for plotting etc.). For example, |
628 - | |
629 - | myflagd = flagcmd(vis=msfile,useapplied=True,action='extract') |
630 - | |
631 - | will extract all the commands (including those already applied) |
632 - | in the FLAG_CMD MS table. |
633 - | |
634 - | NOTE: There is no extant description of the format of this |
635 - | dictionary, as it is an internal device used by the flagcmd |
636 - | task. This action is provided for the convenience of |
637 - | advanced users. |
638 - | |
639 - | |
640 - | ------- FLAG COMMAND SYNTAX ------- |
641 - | |
642 - | The command syntax is based on the flagdata parameters. |
643 - | |
644 - | Basic Syntax Rules |
645 - | |
646 - | Commands are a string (which may contain internal "strings") consisting of |
647 - | KEY=VALUE pairs separated by whitespace (see examples below). |
648 - | |
649 - | NOTE: There should be no whitespace between KEY=VALUE or within each KEY or |
650 - | VALUE, since the simple parser first breaks command lines on whitespace, |
651 - | then on "=". |
652 - | |
653 - | Each key should only appear once on a given command line/string |
654 - | |
655 - | There is an implicit "mode" for each command, with the default |
656 - | being 'manual' if not given. |
657 - | |
658 - | Comment lines can start with '#' and will be ignored. |
659 - | |
660 - | |
661 - | 1. Data selection parameters (used by all flagging modes) |
662 - | |
663 - | timerange='' |
664 - | antenna='' |
665 - | spw='' |
666 - | correlation='' |
667 - | field='' |
668 - | scan='' |
669 - | feed='' |
670 - | array='' |
671 - | uvrange='' |
672 - | intent='' |
673 - | observation='' |
674 - | |
675 - | Note: a command consisting only of selection key-value pairs is a |
676 - | basic "manual" operation, ie. flag the data meeting the selection. |
677 - | |
678 - | |
679 - | 2. Modes specific parameters with default values (for further details and updated |
680 - | default values, refer to the task flagdata). |
681 - | |
682 - | 2.1 Mode manual. |
683 - | autocorr=False |
684 - | |
685 - | 2.2 Mode clip. |
686 - | datacolumn='DATA' |
687 - | clipminmax=[] |
688 - | clipoutside=True |
689 - | channelavg=False |
690 - | clipzeros=False |
691 - | |
692 - | 2.3 Mode shadow. |
693 - | tolerance=0.0 |
694 - | addantenna='' |
695 - | |
696 - | 2.4 Mode quack. |
697 - | quackinterval=1.0 |
698 - | quackmode='beg' |
699 - | quackincrement=False |
700 - | |
701 - | 2.5 Mode elevation. |
702 - | lowerlimit=0.0 |
703 - | upperlimit=90.0 |
704 - | |
705 - | 2.6 Mode tfcrop. |
706 - | ntime='scan' |
707 - | combinescans=False |
708 - | datacolumn='DATA' |
709 - | timecutoff=4.0 |
710 - | freqcutoff=3.0 |
711 - | timefit='line' |
712 - | freqfit='poly' |
713 - | maxnpieces=7 |
714 - | flagdimension='freqtime' |
715 - | usewindowstats='none' |
716 - | halfwin=1 |
717 - | extendflags=True |
718 - | |
719 - | 2.7 Mode extend. |
720 - | ntime='scan' |
721 - | combinescans=False |
722 - | extendpols=True |
723 - | growtime=50.0 |
724 - | growfreq=50.0 |
725 - | growaround=False |
726 - | flagneartime=False |
727 - | flagnearfreq=False |
728 - | |
729 - | 2.8 Mode rflag. |
730 - | ntime='scan' |
731 - | combinescans=False |
732 - | datacolumn='DATA' |
733 - | winsize=3 |
734 - | timedev='' |
735 - | freqdev='' |
736 - | timedevscale=5.0 |
737 - | freqdevscale=5.0 |
738 - | spectralmax=1000000.0 |
739 - | spectralmin=0.0 |
740 - | extendflags=True |
741 - | |
742 - | 2.9 Mode unflag. |
743 - | |
744 - | |
745 - | 3. Basic elaboration options for online and interface use |
746 - | |
747 - | id='' # flag ID tag (not necessary) |
748 - | reason='' # reason string for flag |
749 - | flagtime='' # a timestamp for when this flag was generated (for |
750 - | user history use) |
751 - | |
752 - | NOTE: there is no flagtime column in FLAG_CMD at |
753 - | this time, but we will propose to add this as an |
754 - | optional column |
755 - | |
756 - | NOTE: These are currently ignored and not used. |
757 - | |
758 - | 4. Extended elaboration options for online and interface use |
759 - | Note: these are FLAG_CMD columns, but their use is not clear but included |
760 - | here for compatibility and future expansion |
761 - | |
762 - | level=N # flagging "level" for flags with same reason |
763 - | severity=N # Severity code for the flag, on a scale of 0-10 in order |
764 - | of increasing severity; user specified |
765 - | |
381 + | https://casa.nrao.edu/casadocs/ |
766 382 | |
767 383 | </example> |
768 - | </task> |
384 + | </task> |
769 385 | </casaxml> |