What to Expect When You are Expecting a New Hutch Python

The purpose of this tutorial is to provide a general overview of the new hutch_python system. Underneath the code is built on top of ophyd and bluesky as well as many PCDS specific libraries. If you are looking for more in depth information on a specific part of the control system documentation is hosted on https://pcdshub.github.io.

Table of Contents

If you do not want to setup a Jupyter Notebook environment yourself, skip ahead to navigation. The code below is executed automatically when you launch the hutch python environment at a hutch.

[1]:
%matplotlib notebook

[2]:
from hutch_python.load_conf import load
from hutch_python.log_setup import setup_logging, debug_mode
from pcdsdaq.sim import set_sim_mode as set_daq_sim
from bluesky.utils import install_nb_kicker

[3]:
setup_logging(dir_logs='logs')
install_nb_kicker()
set_daq_sim(True)
globals().update(load('conf.yml'))
            __      _____       _   _
           / _|    |  __ \     | | | |
 _ __ ___ | |___  _| |__) |   _| |_| |__   ___  _ __
| '_ ` _ \|  _\ \/ /  ___/ | | | __| '_ \ / _ \| '_ \
| | | | | | |  >  <| |   | |_| | |_| | | | (_) | | | |
|_| |_| |_|_| /_/\_\_|    \__, |\__|_| |_|\___/|_| |_|
                           __/ |
                          |___/

INFO     Loading daq...
SUCCESS  Successfully loaded daq
INFO     Loading database...
INFO     Loading mfx_mxt_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_attenuator [pcdsdevices.device_types.Attenuator] ... SUCCESS!
INFO     Loading mfx_dg1_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO     Loading mfx_dg1_pim [pcdsdevices.device_types.PIM] ... SUCCESS!
INFO     Loading mfx_dg1_valve_1 [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_dg1_valve_2 [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_dg2_downstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO     Loading mfx_dg2_midstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO     Loading mfx_dg2_upstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO     Loading mfx_dg2_pim [pcdsdevices.device_types.PIM] ... SUCCESS!
INFO     Loading mfx_pulsepicker [pcdsdevices.device_types.PulsePicker] ... SUCCESS!
INFO     Loading mfx_dia_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_dg2_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_dvd_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO     Loading mfx_dia_ipm [pcdsdevices.device_types.IPM] ... SUCCESS!
INFO     Loading mfx_dia_pim [pcdsdevices.pim.PIMMotor] ... SUCCESS!
INFO     Loading mfx_reflaser [pcdsdevices.inout.InOutRecordPositioner] ... SUCCESS!
INFO     Loading sh45 [pcdsdevices.valve.PPSStopper] ... SUCCESS!
SUCCESS  Successfully loaded database
INFO     Loading mfx.beamline...
INFO     Loading transfocator...
SUCCESS  Successfully loaded transfocator
INFO     Loading mfx_prefocus...
SUCCESS  Successfully loaded mfx_prefocus
INFO     Loading beam_suspender...
SUCCESS  Successfully loaded beam_suspender
SUCCESS  Successfully loaded mfx.beamline
INFO     Loading questionnaire...
WARNING  Unable to create an object from Questionnaire table motors row 10
WARNING  Unable to create an object from Questionnaire table motors row 11
INFO     Loading laser_waveplate [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading inj_x [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading inj_y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading shield_y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading shield_x [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading off_nav_focus [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO     Loading off_nav_zoom [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO     Loading tilt [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading tip [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading onAxis_Y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading onAxis_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading offAxis_Y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading offAxis_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading platform_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO     Loading in_nav_zoom [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO     Loading in_nav_focus [pcdsdevices.epics_motor.IMS] ... SUCCESS!
SUCCESS  Successfully loaded questionnaire
INFO     Loading ls2116...
SUCCESS  Successfully loaded ls2116
INFO     Loading default groups...
SUCCESS  Successfully loaded default groups

How Devices Are Loaded

Individual EPICS devices can come three different locations within the hutch_python ecosystem; the permenant happi device database, the LCLS User Questionnaire, and the mfx repository itself. By watching the loading banner itself you should be able to tell which device came from where. In general, static beamline instrumentation should be held within the happi database. This gives us a stable understanding of most of the beamline devices and will also be used to inform other user interfaces and applications in the future.

Included is information on where the device is, what the prefix of the camera is e.t.c. Devices loaded this way will have an attribute .md that provides a helpful link to the database entry behind the device.

[10]:
mfx_dg1_pim.md.show_info()
+--------------+------------------------------------------------------+
| EntryInfo    | Value                                                |
+--------------+------------------------------------------------------+
| active       | True                                                 |
| args         | ['{{prefix}}']                                       |
| beamline     | MFX                                                  |
| creation     | Mon Jan 29 14:04:28 2018                             |
| data         | None                                                 |
| device_class | pcdsdevices.device_types.PIM                         |
| kwargs       | {'name': '{{name}}', 'prefix_det': '{{prefix_det}}'} |
| last_edit    | Tue Feb 27 10:21:22 2018                             |
| macros       | None                                                 |
| name         | mfx_dg1_pim                                          |
| parent       | None                                                 |
| prefix       | MFX:DG1:PIM                                          |
| prefix_det   | MFX:DG1:P6740                                        |
| screen       | None                                                 |
| stand        | DG1                                                  |
| system       | diagnostic                                           |
| type         | PIM                                                  |
| z            | 1019.79000                                           |
+--------------+------------------------------------------------------+

The LCLS Questionnaire will produce similar looking devices. The difference is that these are loaded directly from the information contained on the PCDS setup page. This means if you would like to change any of these devices simply update the appropriate table and reload your session. The proposal number is retrieved automatically but if you would like to force the configuration to use a different proposal’s setup simply specify it in the conf.yaml

experiment:
     proposal: LS21
     run: 16
[11]:
shield_x.md.show_info()
+--------------+---------------------------------+
| EntryInfo    | Value                           |
+--------------+---------------------------------+
| active       | True                            |
| args         | ['{{prefix}}']                  |
| beamline     | MFX                             |
| device_class | pcdsdevices.epics_motor.Newport |
| kwargs       | {'name': '{{name}}'}            |
| location     | Hutch-main experimental         |
| macros       | None                            |
| name         | shield_x                        |
| parent       | None                            |
| prefix       | MFX:USR:MMN:32                  |
| purpose      | X-ray Shield Tube X             |
| screen       | None                            |
| stand        | None                            |
| system       | None                            |
| type         | Device                          |
| z            | -1.00000                        |
+--------------+---------------------------------+

Now if devices aren’t loaded through either the Questionnaire or the device database itself, the old model of placing objects in an experiment specific file or the hutch/beamline.py file are still supported. These maybe device types that are still under development and have not been pushed upstream e.t.c

An Introduction to Scanning

One of the major additions the hutch_python restructuring gives us is access to bluesky scanning capabilities. A full tutorial is available at http://nsls-ii.github.io/bluesky/tutorial. This very brief introduction assumes you know a little about how bluesky works in general. The major keys are:

  • The RunEngine object is responsible for executing all scans.

  • Experimental procedures are described in plans, python generators which allow sophisticated flow control

The hutch-python environment should already have a RunEngine instatiated for you to begin playing. It also has built-in bluesky plans that are tab-accessible in the following objects:

  • bp: full plans that are ready to use

  • bps: partial plans that can be used as building blocks for larger plans

  • bpp: wrappers that add functionality to existing plans

Note

In the examples below we run our scans with simulated hardware built-in to the ophyd library.

[12]:
from ophyd.sim import motor, det

The simplest plan in bluesky is count, a simple reading of a detector. A basic execution looks like this where we take 5 measurements from the detector. Take note what is actually happenning here. We create an instance of count then we pass this into RE().

[13]:
RE(bp.count([det], num=5))
Transient Scan ID: 1     Time: 2018/03/14 09:18:37
Persistent Unique Scan ID: 'dda5fba6-989a-4069-9aa3-d40415534e4c'
New stream: 'primary'
+-----------+------------+------------+
|   seq_num |       time |        det |
+-----------+------------+------------+
|         1 | 09:18:37.9 |      1.000 |
|         2 | 09:18:38.1 |      1.000 |
|         3 | 09:18:38.1 |      1.000 |
|         4 | 09:18:38.1 |      1.000 |
|         5 | 09:18:38.1 |      1.000 |
+-----------+------------+------------+
generator count ['dda5fba6'] (scan num: 1)



[13]:
('dda5fba6-989a-4069-9aa3-d40415534e4c',)

Slightly more interesting is a basic scan where we observe a fixed set of points of a motor reading our detector at each point. If you don’t have a detector to read you can just pass in an empty list. The following will scan between -5, 5 in 10 steps.

[14]:
plan = bp.scan([det], motor, -5, 5, num=10)
RE(plan)
Transient Scan ID: 2     Time: 2018/03/14 09:18:38
Persistent Unique Scan ID: '8f13258c-1c09-41e9-a774-97f3a56a0e1c'
New stream: 'primary'
+-----------+------------+------------+------------+
|   seq_num |       time |      motor |        det |
+-----------+------------+------------+------------+
|         1 | 09:18:38.3 |     -5.000 |      0.000 |
|         2 | 09:18:38.4 |     -3.889 |      0.001 |
|         3 | 09:18:38.4 |     -2.778 |      0.021 |
|         4 | 09:18:38.4 |     -1.667 |      0.249 |
|         5 | 09:18:38.4 |     -0.556 |      0.857 |
|         6 | 09:18:38.4 |      0.556 |      0.857 |
|         7 | 09:18:38.5 |      1.667 |      0.249 |
|         8 | 09:18:38.5 |      2.778 |      0.021 |
|         9 | 09:18:38.5 |      3.889 |      0.001 |
|        10 | 09:18:38.5 |      5.000 |      0.000 |
+-----------+------------+------------+------------+
generator scan ['8f13258c'] (scan num: 2)



[14]:
('8f13258c-1c09-41e9-a774-97f3a56a0e1c',)

More complex scans are possible including adaptive scans where areas of interest receive a higher density of measurements

[15]:
RE(bp.adaptive_scan([det], 'det', motor,
                    start=-15,
                    stop=10,
                    min_step=0.01,
                    max_step=5,
                    target_delta=.05,
                    backstep=True))
Transient Scan ID: 3     Time: 2018/03/14 09:18:38
Persistent Unique Scan ID: '4cde65ae-84b9-47f6-94b7-14d0331e42b4'
New stream: 'primary'
+-----------+------------+------------+------------+
|   seq_num |       time |      motor |        det |
+-----------+------------+------------+------------+
|         1 | 09:18:38.7 |    -15.000 |      0.000 |
|         2 | 09:18:38.7 |    -12.505 |      0.000 |
|         3 | 09:18:38.8 |     -9.509 |      0.000 |
|         4 | 09:18:38.8 |     -6.112 |      0.000 |
|         5 | 09:18:38.8 |     -2.395 |      0.057 |
|         6 | 09:18:38.8 |      1.233 |      0.468 |
|         7 | 09:18:38.8 |     -1.953 |      0.148 |
|         8 | 09:18:38.8 |     -2.154 |      0.098 |
|         9 | 09:18:38.8 |     -1.903 |      0.164 |
|        10 | 09:18:38.8 |     -1.961 |      0.146 |
|        11 | 09:18:38.8 |     -1.767 |      0.210 |
|        12 | 09:18:38.8 |     -1.809 |      0.195 |
|        13 | 09:18:38.8 |     -1.656 |      0.254 |
|        14 | 09:18:38.9 |     -1.507 |      0.321 |
|        15 | 09:18:38.9 |     -1.545 |      0.303 |
|        16 | 09:18:38.9 |     -1.435 |      0.357 |
|        17 | 09:18:38.9 |     -1.325 |      0.415 |
|        18 | 09:18:38.9 |     -1.219 |      0.475 |
|        19 | 09:18:39.0 |     -1.117 |      0.536 |
|        20 | 09:18:39.0 |     -1.018 |      0.596 |
|        21 | 09:18:39.0 |     -0.922 |      0.654 |
|        22 | 09:18:39.0 |     -0.829 |      0.709 |
|        23 | 09:18:39.0 |     -0.738 |      0.762 |
|        24 | 09:18:39.0 |     -0.647 |      0.811 |
|        25 | 09:18:39.0 |     -0.557 |      0.856 |
|        26 | 09:18:39.0 |     -0.464 |      0.898 |
|        27 | 09:18:39.0 |     -0.368 |      0.935 |
|        28 | 09:18:39.0 |     -0.265 |      0.966 |
|        29 | 09:18:39.0 |     -0.149 |      0.989 |
|        30 | 09:18:39.1 |     -0.007 |      1.000 |
|        31 | 09:18:39.1 |      0.236 |      0.972 |
|        32 | 09:18:39.1 |      0.519 |      0.874 |
|        33 | 09:18:39.1 |      0.380 |      0.930 |
|        34 | 09:18:39.1 |      0.529 |      0.870 |
|        35 | 09:18:39.1 |      0.672 |      0.798 |
|        36 | 09:18:39.2 |      0.629 |      0.821 |
|        37 | 09:18:39.2 |      0.729 |      0.767 |
|        38 | 09:18:39.2 |      0.828 |      0.710 |
|        39 | 09:18:39.2 |      0.925 |      0.652 |
|        40 | 09:18:39.2 |      1.018 |      0.595 |
|        41 | 09:18:39.2 |      1.110 |      0.540 |
|        42 | 09:18:39.2 |      1.200 |      0.487 |
|        43 | 09:18:39.2 |      1.289 |      0.436 |
|        44 | 09:18:39.3 |      1.377 |      0.387 |
|        45 | 09:18:39.3 |      1.466 |      0.341 |
|        46 | 09:18:39.3 |      1.557 |      0.298 |
|        47 | 09:18:39.3 |      1.650 |      0.256 |
|        48 | 09:18:39.3 |      1.747 |      0.217 |
|        49 | 09:18:39.3 |      1.850 |      0.181 |
+-----------+------------+------------+------------+
|   seq_num |       time |      motor |        det |
+-----------+------------+------------+------------+
|        50 | 09:18:39.3 |      1.960 |      0.146 |
|        51 | 09:18:39.3 |      2.080 |      0.115 |
|        52 | 09:18:39.3 |      2.215 |      0.086 |
|        53 | 09:18:39.3 |      2.369 |      0.060 |
|        54 | 09:18:39.4 |      2.552 |      0.039 |
|        55 | 09:18:39.4 |      2.783 |      0.021 |
|        56 | 09:18:39.4 |      3.097 |      0.008 |
|        57 | 09:18:39.4 |      3.599 |      0.002 |
|        58 | 09:18:39.4 |      4.747 |      0.000 |
|        59 | 09:18:39.4 |      6.666 |      0.000 |
|        60 | 09:18:39.5 |      9.200 |      0.000 |
+-----------+------------+------------+------------+
generator adaptive_scan ['4cde65ae'] (scan num: 3)



[15]:
('4cde65ae-84b9-47f6-94b7-14d0331e42b4',)

Including the DAQ

What has been ignored in these examples is the inclusion of the DAQ. There are a few simple helper functions that cover basic modes of operations. The simplest behavior is just running the DAQ throughout the whole scan and stopping at the end. This can be accomlished by passing any plan you want to run this way through the daq_wrapper

[16]:
RE(bpp.daq_wrapper(bp.scan([det], motor, -5, 5, num=10)))
INFO     Daq configured
Transient Scan ID: 4     Time: 2018/03/14 09:18:39
Persistent Unique Scan ID: 'cd9c1873-4519-48d2-bd88-21cd1a7d9c53'
New stream: 'primary'
+-----------+------------+------------+------------+
|   seq_num |       time |      motor |        det |
+-----------+------------+------------+------------+
|         1 | 09:18:39.7 |     -5.000 |      0.000 |
|         2 | 09:18:39.7 |     -3.889 |      0.001 |
|         3 | 09:18:39.7 |     -2.778 |      0.021 |
|         4 | 09:18:39.7 |     -1.667 |      0.249 |
|         5 | 09:18:39.7 |     -0.556 |      0.857 |
|         6 | 09:18:39.8 |      0.556 |      0.857 |
|         7 | 09:18:39.8 |      1.667 |      0.249 |
|         8 | 09:18:39.8 |      2.778 |      0.021 |
|         9 | 09:18:39.8 |      3.889 |      0.001 |
|        10 | 09:18:39.9 |      5.000 |      0.000 |
+-----------+------------+------------+------------+
generator scan ['cd9c1873'] (scan num: 4)



[16]:
('cd9c1873-4519-48d2-bd88-21cd1a7d9c53',)

More complex behavior is available. You can have the daq run at every scan step by passing it into a scan instead of or in addition to a det input from any of the previous examples. The most common mode of operation for this mode is when performing calibration cycles. You can also make the daq run at very specific times by writing your own plans that do yield from bps.trigger_and_read(daq).

[17]:
daq.configure(events=120)
RE(bp.scan([daq, det], motor, -5, 5, num=10))
INFO     Daq configured
Transient Scan ID: 5     Time: 2018/03/14 09:18:40
Persistent Unique Scan ID: 'c4218a45-7ce5-4655-b35a-1bd85174e93d'
New stream: 'primary'
+-----------+------------+------------+------------+
|   seq_num |       time |      motor |        det |
+-----------+------------+------------+------------+
|         1 | 09:18:40.1 |     -5.000 |      0.000 |
|         2 | 09:18:40.6 |     -3.889 |      0.001 |
|         3 | 09:18:41.1 |     -2.778 |      0.021 |
|         4 | 09:18:41.7 |     -1.667 |      0.249 |
|         5 | 09:18:42.2 |     -0.556 |      0.857 |
|         6 | 09:18:42.7 |      0.556 |      0.857 |
|         7 | 09:18:43.2 |      1.667 |      0.249 |
|         8 | 09:18:43.7 |      2.778 |      0.021 |
|         9 | 09:18:44.2 |      3.889 |      0.001 |
|        10 | 09:18:44.7 |      5.000 |      0.000 |
+-----------+------------+------------+------------+
generator scan ['c4218a45'] (scan num: 5)



[17]:
('c4218a45-7ce5-4655-b35a-1bd85174e93d',)