What to Expect When You are Expecting a New Hutch Python¶
The purpose of this tutorial is to provide a general overview of the new hutch_python
system. Underneath the code is built on top of ophyd and bluesky as well as many PCDS specific libraries. If you are looking for more in depth information on a specific part of the control system documentation is hosted on https://pcdshub.github.io.
Table of Contents¶
If you do not want to setup a Jupyter Notebook environment yourself, skip ahead to navigation. The code below is executed automatically when you launch the hutch python environment at a hutch.
[1]:
%matplotlib notebook
[2]:
from hutch_python.load_conf import load
from hutch_python.log_setup import setup_logging, debug_mode
from pcdsdaq.sim import set_sim_mode as set_daq_sim
from bluesky.utils import install_nb_kicker
[3]:
setup_logging(dir_logs='logs')
install_nb_kicker()
set_daq_sim(True)
globals().update(load('conf.yml'))
__ _____ _ _
/ _| | __ \ | | | |
_ __ ___ | |___ _| |__) | _| |_| |__ ___ _ __
| '_ ` _ \| _\ \/ / ___/ | | | __| '_ \ / _ \| '_ \
| | | | | | | > <| | | |_| | |_| | | | (_) | | | |
|_| |_| |_|_| /_/\_\_| \__, |\__|_| |_|\___/|_| |_|
__/ |
|___/
INFO Loading daq...
SUCCESS Successfully loaded daq
INFO Loading database...
INFO Loading mfx_mxt_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_attenuator [pcdsdevices.device_types.Attenuator] ... SUCCESS!
INFO Loading mfx_dg1_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO Loading mfx_dg1_pim [pcdsdevices.device_types.PIM] ... SUCCESS!
INFO Loading mfx_dg1_valve_1 [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_dg1_valve_2 [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_dg2_downstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO Loading mfx_dg2_midstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO Loading mfx_dg2_upstream_slits [pcdsdevices.device_types.Slits] ... SUCCESS!
INFO Loading mfx_dg2_pim [pcdsdevices.device_types.PIM] ... SUCCESS!
INFO Loading mfx_pulsepicker [pcdsdevices.device_types.PulsePicker] ... SUCCESS!
INFO Loading mfx_dia_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_dg2_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_dvd_valve [pcdsdevices.device_types.GateValve] ... SUCCESS!
INFO Loading mfx_dia_ipm [pcdsdevices.device_types.IPM] ... SUCCESS!
INFO Loading mfx_dia_pim [pcdsdevices.pim.PIMMotor] ... SUCCESS!
INFO Loading mfx_reflaser [pcdsdevices.inout.InOutRecordPositioner] ... SUCCESS!
INFO Loading sh45 [pcdsdevices.valve.PPSStopper] ... SUCCESS!
SUCCESS Successfully loaded database
INFO Loading mfx.beamline...
INFO Loading transfocator...
SUCCESS Successfully loaded transfocator
INFO Loading mfx_prefocus...
SUCCESS Successfully loaded mfx_prefocus
INFO Loading beam_suspender...
SUCCESS Successfully loaded beam_suspender
SUCCESS Successfully loaded mfx.beamline
INFO Loading questionnaire...
WARNING Unable to create an object from Questionnaire table motors row 10
WARNING Unable to create an object from Questionnaire table motors row 11
INFO Loading laser_waveplate [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading inj_x [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading inj_y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading shield_y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading shield_x [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading off_nav_focus [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO Loading off_nav_zoom [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO Loading tilt [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading tip [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading onAxis_Y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading onAxis_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading offAxis_Y [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading offAxis_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading platform_Z [pcdsdevices.epics_motor.Newport] ... SUCCESS!
INFO Loading in_nav_zoom [pcdsdevices.epics_motor.IMS] ... SUCCESS!
INFO Loading in_nav_focus [pcdsdevices.epics_motor.IMS] ... SUCCESS!
SUCCESS Successfully loaded questionnaire
INFO Loading ls2116...
SUCCESS Successfully loaded ls2116
INFO Loading default groups...
SUCCESS Successfully loaded default groups
How Devices Are Loaded¶
Individual EPICS devices can come three different locations within the hutch_python
ecosystem; the permenant happi
device database, the LCLS User Questionnaire, and the mfx
repository itself. By watching the loading banner itself you should be able to tell which device came from where. In general, static beamline instrumentation should be held within the happi
database. This gives us a stable understanding of most of the beamline devices and will also be used to inform other user
interfaces and applications in the future.
Included is information on where the device is, what the prefix of the camera is e.t.c. Devices loaded this way will have an attribute .md
that provides a helpful link to the database entry behind the device.
[10]:
mfx_dg1_pim.md.show_info()
+--------------+------------------------------------------------------+
| EntryInfo | Value |
+--------------+------------------------------------------------------+
| active | True |
| args | ['{{prefix}}'] |
| beamline | MFX |
| creation | Mon Jan 29 14:04:28 2018 |
| data | None |
| device_class | pcdsdevices.device_types.PIM |
| kwargs | {'name': '{{name}}', 'prefix_det': '{{prefix_det}}'} |
| last_edit | Tue Feb 27 10:21:22 2018 |
| macros | None |
| name | mfx_dg1_pim |
| parent | None |
| prefix | MFX:DG1:PIM |
| prefix_det | MFX:DG1:P6740 |
| screen | None |
| stand | DG1 |
| system | diagnostic |
| type | PIM |
| z | 1019.79000 |
+--------------+------------------------------------------------------+
The LCLS Questionnaire will produce similar looking devices. The difference is that these are loaded directly from the information contained on the PCDS setup page. This means if you would like to change any of these devices simply update the appropriate table and reload your session. The proposal number is retrieved automatically but if you would like to force the configuration to use a different proposal’s setup simply specify it in the conf.yaml
experiment:
proposal: LS21
run: 16
[11]:
shield_x.md.show_info()
+--------------+---------------------------------+
| EntryInfo | Value |
+--------------+---------------------------------+
| active | True |
| args | ['{{prefix}}'] |
| beamline | MFX |
| device_class | pcdsdevices.epics_motor.Newport |
| kwargs | {'name': '{{name}}'} |
| location | Hutch-main experimental |
| macros | None |
| name | shield_x |
| parent | None |
| prefix | MFX:USR:MMN:32 |
| purpose | X-ray Shield Tube X |
| screen | None |
| stand | None |
| system | None |
| type | Device |
| z | -1.00000 |
+--------------+---------------------------------+
Now if devices aren’t loaded through either the Questionnaire or the device database itself, the old model of placing objects in an experiment specific file or the hutch/beamline.py
file are still supported. These maybe device types that are still under development and have not been pushed upstream e.t.c
An Introduction to Scanning¶
One of the major additions the hutch_python
restructuring gives us is access to bluesky
scanning capabilities. A full tutorial is available at http://nsls-ii.github.io/bluesky/tutorial. This very brief introduction assumes you know a little about how bluesky
works in general. The major keys are:
The
RunEngine
object is responsible for executing all scans.Experimental procedures are described in
plans
, python generators which allow sophisticated flow control
The hutch-python environment should already have a RunEngine
instatiated for you to begin playing. It also has built-in bluesky
plans
that are tab-accessible in the following objects:
bp
: full plans that are ready to usebps
: partial plans that can be used as building blocks for larger plansbpp
: wrappers that add functionality to existing plans
Note¶
In the examples below we run our scans with simulated hardware built-in to the ophyd
library.
[12]:
from ophyd.sim import motor, det
The simplest plan in bluesky
is count
, a simple reading of a detector. A basic execution looks like this where we take 5 measurements from the detector. Take note what is actually happenning here. We create an instance of count
then we pass this into RE()
.
[13]:
RE(bp.count([det], num=5))
Transient Scan ID: 1 Time: 2018/03/14 09:18:37
Persistent Unique Scan ID: 'dda5fba6-989a-4069-9aa3-d40415534e4c'
New stream: 'primary'
+-----------+------------+------------+
| seq_num | time | det |
+-----------+------------+------------+
| 1 | 09:18:37.9 | 1.000 |
| 2 | 09:18:38.1 | 1.000 |
| 3 | 09:18:38.1 | 1.000 |
| 4 | 09:18:38.1 | 1.000 |
| 5 | 09:18:38.1 | 1.000 |
+-----------+------------+------------+
generator count ['dda5fba6'] (scan num: 1)
[13]:
('dda5fba6-989a-4069-9aa3-d40415534e4c',)
Slightly more interesting is a basic scan where we observe a fixed set of points of a motor reading our detector at each point. If you don’t have a detector to read you can just pass in an empty list. The following will scan between -5, 5 in 10 steps.
[14]:
plan = bp.scan([det], motor, -5, 5, num=10)
RE(plan)
Transient Scan ID: 2 Time: 2018/03/14 09:18:38
Persistent Unique Scan ID: '8f13258c-1c09-41e9-a774-97f3a56a0e1c'
New stream: 'primary'
+-----------+------------+------------+------------+
| seq_num | time | motor | det |
+-----------+------------+------------+------------+
| 1 | 09:18:38.3 | -5.000 | 0.000 |
| 2 | 09:18:38.4 | -3.889 | 0.001 |
| 3 | 09:18:38.4 | -2.778 | 0.021 |
| 4 | 09:18:38.4 | -1.667 | 0.249 |
| 5 | 09:18:38.4 | -0.556 | 0.857 |
| 6 | 09:18:38.4 | 0.556 | 0.857 |
| 7 | 09:18:38.5 | 1.667 | 0.249 |
| 8 | 09:18:38.5 | 2.778 | 0.021 |
| 9 | 09:18:38.5 | 3.889 | 0.001 |
| 10 | 09:18:38.5 | 5.000 | 0.000 |
+-----------+------------+------------+------------+
generator scan ['8f13258c'] (scan num: 2)
[14]:
('8f13258c-1c09-41e9-a774-97f3a56a0e1c',)
More complex scans are possible including adaptive scans where areas of interest receive a higher density of measurements
[15]:
RE(bp.adaptive_scan([det], 'det', motor,
start=-15,
stop=10,
min_step=0.01,
max_step=5,
target_delta=.05,
backstep=True))
Transient Scan ID: 3 Time: 2018/03/14 09:18:38
Persistent Unique Scan ID: '4cde65ae-84b9-47f6-94b7-14d0331e42b4'
New stream: 'primary'
+-----------+------------+------------+------------+
| seq_num | time | motor | det |
+-----------+------------+------------+------------+
| 1 | 09:18:38.7 | -15.000 | 0.000 |
| 2 | 09:18:38.7 | -12.505 | 0.000 |
| 3 | 09:18:38.8 | -9.509 | 0.000 |
| 4 | 09:18:38.8 | -6.112 | 0.000 |
| 5 | 09:18:38.8 | -2.395 | 0.057 |
| 6 | 09:18:38.8 | 1.233 | 0.468 |
| 7 | 09:18:38.8 | -1.953 | 0.148 |
| 8 | 09:18:38.8 | -2.154 | 0.098 |
| 9 | 09:18:38.8 | -1.903 | 0.164 |
| 10 | 09:18:38.8 | -1.961 | 0.146 |
| 11 | 09:18:38.8 | -1.767 | 0.210 |
| 12 | 09:18:38.8 | -1.809 | 0.195 |
| 13 | 09:18:38.8 | -1.656 | 0.254 |
| 14 | 09:18:38.9 | -1.507 | 0.321 |
| 15 | 09:18:38.9 | -1.545 | 0.303 |
| 16 | 09:18:38.9 | -1.435 | 0.357 |
| 17 | 09:18:38.9 | -1.325 | 0.415 |
| 18 | 09:18:38.9 | -1.219 | 0.475 |
| 19 | 09:18:39.0 | -1.117 | 0.536 |
| 20 | 09:18:39.0 | -1.018 | 0.596 |
| 21 | 09:18:39.0 | -0.922 | 0.654 |
| 22 | 09:18:39.0 | -0.829 | 0.709 |
| 23 | 09:18:39.0 | -0.738 | 0.762 |
| 24 | 09:18:39.0 | -0.647 | 0.811 |
| 25 | 09:18:39.0 | -0.557 | 0.856 |
| 26 | 09:18:39.0 | -0.464 | 0.898 |
| 27 | 09:18:39.0 | -0.368 | 0.935 |
| 28 | 09:18:39.0 | -0.265 | 0.966 |
| 29 | 09:18:39.0 | -0.149 | 0.989 |
| 30 | 09:18:39.1 | -0.007 | 1.000 |
| 31 | 09:18:39.1 | 0.236 | 0.972 |
| 32 | 09:18:39.1 | 0.519 | 0.874 |
| 33 | 09:18:39.1 | 0.380 | 0.930 |
| 34 | 09:18:39.1 | 0.529 | 0.870 |
| 35 | 09:18:39.1 | 0.672 | 0.798 |
| 36 | 09:18:39.2 | 0.629 | 0.821 |
| 37 | 09:18:39.2 | 0.729 | 0.767 |
| 38 | 09:18:39.2 | 0.828 | 0.710 |
| 39 | 09:18:39.2 | 0.925 | 0.652 |
| 40 | 09:18:39.2 | 1.018 | 0.595 |
| 41 | 09:18:39.2 | 1.110 | 0.540 |
| 42 | 09:18:39.2 | 1.200 | 0.487 |
| 43 | 09:18:39.2 | 1.289 | 0.436 |
| 44 | 09:18:39.3 | 1.377 | 0.387 |
| 45 | 09:18:39.3 | 1.466 | 0.341 |
| 46 | 09:18:39.3 | 1.557 | 0.298 |
| 47 | 09:18:39.3 | 1.650 | 0.256 |
| 48 | 09:18:39.3 | 1.747 | 0.217 |
| 49 | 09:18:39.3 | 1.850 | 0.181 |
+-----------+------------+------------+------------+
| seq_num | time | motor | det |
+-----------+------------+------------+------------+
| 50 | 09:18:39.3 | 1.960 | 0.146 |
| 51 | 09:18:39.3 | 2.080 | 0.115 |
| 52 | 09:18:39.3 | 2.215 | 0.086 |
| 53 | 09:18:39.3 | 2.369 | 0.060 |
| 54 | 09:18:39.4 | 2.552 | 0.039 |
| 55 | 09:18:39.4 | 2.783 | 0.021 |
| 56 | 09:18:39.4 | 3.097 | 0.008 |
| 57 | 09:18:39.4 | 3.599 | 0.002 |
| 58 | 09:18:39.4 | 4.747 | 0.000 |
| 59 | 09:18:39.4 | 6.666 | 0.000 |
| 60 | 09:18:39.5 | 9.200 | 0.000 |
+-----------+------------+------------+------------+
generator adaptive_scan ['4cde65ae'] (scan num: 3)
[15]:
('4cde65ae-84b9-47f6-94b7-14d0331e42b4',)
Including the DAQ¶
What has been ignored in these examples is the inclusion of the DAQ. There are a few simple helper functions that cover basic modes of operations. The simplest behavior is just running the DAQ throughout the whole scan and stopping at the end. This can be accomlished by passing any plan you want to run this way through the daq_wrapper
[16]:
RE(bpp.daq_wrapper(bp.scan([det], motor, -5, 5, num=10)))
INFO Daq configured
Transient Scan ID: 4 Time: 2018/03/14 09:18:39
Persistent Unique Scan ID: 'cd9c1873-4519-48d2-bd88-21cd1a7d9c53'
New stream: 'primary'
+-----------+------------+------------+------------+
| seq_num | time | motor | det |
+-----------+------------+------------+------------+
| 1 | 09:18:39.7 | -5.000 | 0.000 |
| 2 | 09:18:39.7 | -3.889 | 0.001 |
| 3 | 09:18:39.7 | -2.778 | 0.021 |
| 4 | 09:18:39.7 | -1.667 | 0.249 |
| 5 | 09:18:39.7 | -0.556 | 0.857 |
| 6 | 09:18:39.8 | 0.556 | 0.857 |
| 7 | 09:18:39.8 | 1.667 | 0.249 |
| 8 | 09:18:39.8 | 2.778 | 0.021 |
| 9 | 09:18:39.8 | 3.889 | 0.001 |
| 10 | 09:18:39.9 | 5.000 | 0.000 |
+-----------+------------+------------+------------+
generator scan ['cd9c1873'] (scan num: 4)
[16]:
('cd9c1873-4519-48d2-bd88-21cd1a7d9c53',)
More complex behavior is available. You can have the daq run at every scan step by passing it into a scan instead of or in addition to a det
input from any of the previous examples. The most common mode of operation for this mode is when performing calibration cycles. You can also make the daq run at very specific times by writing your own plans that do yield from bps.trigger_and_read(daq)
.
[17]:
daq.configure(events=120)
RE(bp.scan([daq, det], motor, -5, 5, num=10))
INFO Daq configured
Transient Scan ID: 5 Time: 2018/03/14 09:18:40
Persistent Unique Scan ID: 'c4218a45-7ce5-4655-b35a-1bd85174e93d'
New stream: 'primary'
+-----------+------------+------------+------------+
| seq_num | time | motor | det |
+-----------+------------+------------+------------+
| 1 | 09:18:40.1 | -5.000 | 0.000 |
| 2 | 09:18:40.6 | -3.889 | 0.001 |
| 3 | 09:18:41.1 | -2.778 | 0.021 |
| 4 | 09:18:41.7 | -1.667 | 0.249 |
| 5 | 09:18:42.2 | -0.556 | 0.857 |
| 6 | 09:18:42.7 | 0.556 | 0.857 |
| 7 | 09:18:43.2 | 1.667 | 0.249 |
| 8 | 09:18:43.7 | 2.778 | 0.021 |
| 9 | 09:18:44.2 | 3.889 | 0.001 |
| 10 | 09:18:44.7 | 5.000 | 0.000 |
+-----------+------------+------------+------------+
generator scan ['c4218a45'] (scan num: 5)
[17]:
('c4218a45-7ce5-4655-b35a-1bd85174e93d',)