March 19, 2013 SITL workshop, Boulder CO:
Team members are welcome to correct errors or add additional information by using the "edit this page" function under the "edit" function above.
More than 50 members of the MMS management, spacecraft, instrument teams attended.
Robert Ergun and Thomas Moore opened the meeting and introduced the goals of the meeting:
- Energize SOC-SDC-SITL Team
- Review Science Team-SOC-SITL R&R
- Identify instrument team’s R&R
- Inventory SITL products, tools, processes
- Plan toward integrated testing of SITL
- Identify actions needed by next SWT mtg.
Lon Riesberg presented the SOC-SITL interface graphics from the SITL ICD document and reviewed the interface plans.
- There was good discussion of the details and renewed agreement on the plans.
- It was noted that the SITL GUI and TPlot representations on the "SITL Tools" portion of the graphic may need some updating to reflect evolution of those tools.
- Min requirement from SITL is for SITL to display CDQs. If Fast Survey data gets down in time, that data would also be displayed. End product from SITL is a list of times in priority order.
- It would also be welcome for the instrument teams to also "give whatever they can give" to help the SITL scientists, e.g., pseudo-moments.
- SITL data retriever and the SITL submitted will be password protected. That protection can be removed later is determined not to be necessary. The plan is to enable that function now.
Bill Paterson reviewed FPI's point-of-view and plans for SITL.
- SITL priorities: (1) initial assessment & tuning of the automated CDQ selection system; (2) routine review of automated CDQ selections; (3) fine tune selections by weighing # of intervals against durations.
- FPI SITL Products:
- Trigger terms and trigger data numbers
- fast survey spectrograms
- fast survey moments
- combined fields/plasma parameters
- Bill acknowledged that compute time is important. FPI benchmarking processing times in response to "push back" of wanting to provide some higher-level data parameters.
- Ergun noted that more folks will want velocities and densities ... survey moments will be highly desired. They don't have to be final determined moments, peudo-moments will be fine. SITL scientists must be trained to understanding the limitations in those moments.
- Bill presented a comprehensive list of parameters the team is working to make available.
- Bill noted that it must be remembered that the FPI trigger terms are "rough equivalents" to the usual moments. They are not properly weighted by energy factors, dot products, angular sections and use integer arithmetic. There are other effects on the computation that change with command mode. The thought is that these types of changes don't happen often. Effects need to be accommodated through the system.
- Bill presented details on how the FPI trigger data numbers are calculated and at what resolution and in what form. These will be at 30ms for DES and 150 ms for DIS. Also, Bill presented notes on the moments that can be calculated on the ground from the Fast Survey data for SITL purposes. These can be at 4.5s resolution and could include some level of calibration.
- Bill notes that computing these psuedo moments will add 5% overhead to the production of the fast survey spectrograms. It is also possible to compute the electron anisotropy, electron agyrotropy, and electron Alfven Mach #. This calculation is thought to be a simple addition. Bob noted that there is no requirement from any team to compute highest level data products.
Roy Torbert presented the Fields Status.
- Roy presented a list of the default trigger data products and the additional SITL products (fast survey resolution for these latter products) and showed how the processing is represented within the Fields team "spaghetti" processing chart.
Bob Ergun presented his graphic on Burst Data Selection from the September 2012 SWT meeting. He noted which parts of the processing have progressed since that meeting. He reminded folks of the three different paths for the burst data to flow and for decisions to be made. The first is the automatic selection and we note that "it must work". The other two paths are based on SITL, one is human-based, one is automated SITL.
Tom Moore asked about staffing. It was noted that there is no additional budget for SITL staffing other than the existing science team. A rotating-by-month SITL scientist has been suggested. SITL scientist must do the work for the agreed time period assisted by a deputy. The deputy learns and then becomes the SITL scientist, and so on. That is a nominal suggestion that the SWG needs to discuss/agree. It was suggested that the PI assigns the SITL scientist/deputy with input from the SWG. Tai Phan explained another method used on THEMIS based on a two-week cycle. It was noted that this could be a big job during the first 6 months. It was noted that the SITL scientist's budget may already be covered by existing funds but any travel would not be. The plan is for travel to be unnecessary. There was a question as to whether one person could handle this job. The answer is that the SITL Scientist is one person, with a team that consists of one person from each team. SITL scientist has about five hours to respond; this will be about once per day on the dayside and about once per three days on the nightside. This is the reason that it is thought this could be a burn-out job.
Lon Riesberg asked about some implementation details that may or may not be compatible with some of the pieces already being developed. Suggested that the burst data telecons start up again to address some of these issues. There was a question about HPCA and EPD involvement; Bob noted that these two instruments are following behind FPI and Fields in this planning. Lon noted that HPCA has a separate processing stream that needs to be worked in the SITL context sooner than later.
UCB Software Demonstration: named EVA, coded with IDL as an extension of TDAS
- Three parts to the software: data handling, data display, and trigger management. SITL computer will need to be able to locally cache the processed data. <-- important to note
- The demonstration by ..... was thorough and smooth. Tom Moore inquired about another style of display. The answer is that any combination of data can be defined and the PI will determined whether support exists for inclusion as a standard plot type. Otherwise, individuals would need to create their own plot type for inclusion into the system. It should be possible for folks to make custom solutions quite easily if familiar with TDAS. Although we don't want this GUI to become terribly cluttered, max of 10 "pull downs".
- See the picture of Tai's office to envision the ideal SITL scientist's office. Theoretically, the SITL scientist should be able to perform their function with a single MacBook Air while on travel.
- There was a question as to when we would be ready to beta-test this system. The answer is that there is still some development to do and testing with some "real" data, like calibration data. Currently working with THEMIS data.
- Folks will need to get TDAS working on their systems and to start working with it. There was a suggestion to prepare a beta-test plan with schedule. There was another suggestion to use Cluster data for beta testing. Others felt that the THEMIS data is better prepared for testing the display software and Cluster is best suited for trigger testing. Again, a beta-test plan is needed to work out who can do what and when. What can be used for initial training?
Russ Panneton presented information on the Burst Selection Cycle. He gave details on the data interfaces with the SOC and the later command interface along with a picture of the SITL "sequence of events".
- An important reminder is the fact that the SITL scientist can look back in time to previous downloads as well as current downloads to monitor what has been captured and ensure that the priorities in the buffer remain as desired.
- How FOM and other parameters will be adjusted with time needs planning.
- Russ pointed out that there will need to be a process for the SITL understand when full data has been received from the set of spacecraft. Data is processed continually regardless, the SITL function needs a method to know when the data is ready for SITL work. There is a plan to download the burst metadata twice to ensure we have it. There would be eight processing cycles then, two per s/c which could happen in any order. There is a suggestion that something/somewhere automatically emails the SITL function that it is time to "pull" the data and then also needs the schedule of deadlines for the "push" of changes to the download priorities. The nominal schedule would be known in advance and will be made available, confirmation emails/texts then need to be disseminated. Also need a process for folks to know who has SITL authority at any given time.
- Good discussion ensured on how the SITL function would proceed in real time. All agreed more thought and documentation was needed on these ideas.
- Russ presented information on the database structure, which will include full information on what transpired along the processing cycle and with time. "I capture footprints of everything we do", and apparently including "who" did it. Bob made a point that we want to capture information on the "basis of decision" for the SITL scientist decisions. If SITL changes some parameters we will want a "comment string" in the data structure to capture this. It will be possible to compare the results of the automated selection system and the SITL selections. How can we make this comparison automatically appear? Either on the SITL display or as a regular report of some kind elsewhere? Method of access is not yet clear at this point for these files or what other diagnostic information may be needed. Bob noted a few files or pieces of information that may be needed on the SITL side.
- Russ presented a list of "things that needs to be done", see the chart in the presentation for a clearer view of this list captured here... this list was from a previous meeting and may not be completely up to date. All should look through and ensure their actions items have been completed or are on track to be completed.
- Documentation --SITL Data Products Guide --SDC-SITL ICD
- SITL Data --Develop, test, maintain software for SITL data production --Create sample SITL products for developing and testing display capabilities, interfaces, data retrieval software, etc.
- SITL Data Access --Develop, test, and maintain interfaces that allow access to SITL data at the SDC --Develop, test, and maintain software on SITL side to interact with the data access interfaces (including local cache management)
- Buffer Selections --Develop, test, and maintain interfaces that allow submissions to the SDC --Develop, test, and maintain software to enable SITL users to create buffer selection structures
Last item on the Agenda:
Actions: Bob noted that there are plenty of details to be worked; some details to the interface diagrams need updating. Lots of fine tuning to do but things appear to be coming together.
NOTES FROM SITL WORKSHOP: items to review/do in the future:
- HPCA processing of data for SITL function
- Representation of SITL Tools portion of the SOC-SITL graphic needs updating
- Communicate the routine CDQ selection algorithms
- How to accommodate the changes that FPI command mode can have on its trigger terms. No free running on changing mode without accommodating through the system. Identify areas this type of thing can occur and write down a process to accommodate. The IS ops lead has a thought on this process.
- Write down SITL staffing plan, request other suggestions (Phan?), circulate, refine, take to SWG.
- Beta testing of SITL system ... need to get the TDAS downloaded, etc. prepare for the next in-person SWG meeting. Prepare a beta test plan.
- Straw poll of folks willing to serve as SITL scientist.
- Review CDF implementation and what metadata will be included to make sure all files are useful to the SITL, and other, functions. Non-standard CDFs are considered to be non-helpful.
- How to notify the SITL that the data is ready for download and when their deadline is for turnaround. Maybe text message? Process for how SITL scientist authority is known.
- Review notes above to see what else needs to be captured in this list ...
- SITL commissioning plan, starting very simple, adding needed complexity as time goes on.
Items for future SWG telecons:
- Review SITL workshop notes - enter corrections/expansion
- Flowchart for Phase D/E Science Communications: updates? agreement?
- SDAOWG telecon schedule, nominal agendas, POCs
- presentation of draft SITL beta test plan: updates? agreement?