Changelog

Here you can see the full list of changes between each Concert release.

Version 0.31

Released on May 11th 2022.

Features

  • it is possible to use await statements in concert sessions outside of async def functions

  • AsyncObject allows us to have async constructors async def __ainit__, which is very useful when we need to use async code in constructors, e.g. setting a parameter on a device

  • Experiments were greatly extended by classes which can be customized and run at beam lines without many more changes

  • Monochromators were enhanced

  • Experiment director was introduced to take care of high-throughput measurements

  • Experiments can write metadata to json

  • Parameter limits getters and setters are coroutine functions

  • Add ElementSelector device for having discrete choices

  • ascan allows one to include the stop value in the scan

  • walkers have create_writer method for convenience

  • walkers can be used in async with statements for safe access from multiple concurrent coroutines

  • concert.coroutines.sinks.Accumulate allows non-re-setting behavior between runs

  • concert.processes.common.align_rotation_axis allows multiple frame consumers

  • concert.devices.cameras.uca.Camera gets state from the libuca object

API breaks

  • Parameterizable inherits from AsyncObject which needs an __ainit__

constructor, so every sub-class must change its __init__ to async def __ainit__. - run_in_loop_thread_blocking is gone in favour of using one event loop and

__ainit__ constructors

Notes

  • we replaced Travis by CircleCI

Pull Requests

#485 from ufo-kit/circleci-project-setup #483 from ufo-kit/new-metadata #464 from ufo-kit/monochromator-enhancement #459 from ufo-kit/experiment-director #409 from ufo-kit/fix-uca-state #467 from ufo-kit/grating-inteferometry-experiment #478 from ufo-kit/hercules #473 from ufo-kit/walkers #471 from ufo-kit/ascan-include-last #470 from ufo-kit/elementselector #466 from ufo-kit/fix-tango-loop #458 from ufo-kit/new-experiments #461 from ufo-kit/async-limits

Version 0.30

  • Simplify scans, no Region and no need of using resolve on the result

Features

  • concurrency transition to asyncio, from user perspective the usage is more or less the same, e.g. motor.move(1 * q.mm) starts relative motion in the background, motor.position = 1 * q.mm sets the position in a blocking way.

  • ctrl-k cancels all background commands and calls emergency_stop on all devices in the session

  • Online reconstruction is capable of doing the flat correction with image averaging

  • concert.experiments.addons.ImageWriter blocks for every acquisition

Viewers

  • Subclass Parameterizeable, properties turned into parameters

  • Added Simple matplotlib backend without colorbar but faster

  • Added PyQtGraphViewer

  • Added downsampling parameter

  • Added show_refresh_rate parameter

  • Parameter limits accepts “stream”, i.e. the limits are updated for the first image of __call__ and then kept the same, “auto” updates limits for every image and a tuple fixes them to min and max.

API breaks

  • Removed queue, sinograms, backproject, PickSlice, process coroutines from concert.coroutines.filters

  • Removed Backproject, FlatCorrectedBackproject, FlatCorrect (was there twice), center_rotation_axis, compute_rotation_axis from concert.ext.ufo

  • Removed concert.networking.aerotech and concert.networking.wago

  • Removed concert.devices.motors.crio

  • Removed concert.helpers.hasattr_raise_exceptions

  • Removed Future.cancel_operation, cancellation is implemented via asyncio.CancelledError

  • Removed concert.helpers.Region

  • Removed devices:
    • concert.devices.motors.aerotech

    • concert.devices.io.aerotech

    • concert.devices.lightsources.fiberlite

    • concert.devices.lightsources.led.

    • concert.devices.photodiodes.edmundoptics

    • concert.devices.scales.adventurer

    • concert.devices.monochromators.newport.py

  • Removed processes:
    • concert.processes.common.find_beam

    • concert.processes.common.drift_to_beam

    • concert.processes.common.center_to_beam

    • concert.processes.common.scan_param_feedback

  • Removed callbacks from concert.processes.common.scan

  • Removed concert.experiments.base.Acquisition.connect for simplicity, i.e. the acquisition and consumption of data happens always at the same time, for special behavior, special classes/functions should be used

  • Device.abort replaced by Device.emergency_stop

  • many functions which were not asynchronous before are now, e.g. Camera.start_recording

Version 0.11

Released on April 14th 2021, which is the last release for Python 2.

Features

  • 345c2e7 Add stack-based name resolution

  • f8e9ec5 experiments: add per-run log handler

  • 41bfa5e Add State to Experiment

  • 253d4d5 Add external limits to parameter

  • 40debd7 Added target value to parameter

  • 6eb07fd Added LensChanger

  • dc0b395 Add readers

  • ec90dcd ufo: Add FlatCorrect class

  • e7326bd Add dummy ImagingExperiment

  • f90379d Add fwhm to find_parameters

  • a8c705b imageprocessing: add filter_low_frequencies

  • e181587 Add a simple online reconstruction addon

  • 51d6ec2 Add ROI selection support to ImagingFileExperiment

  • 2ba2fe1 add fraction and percent to quantities

  • e59697b Add progress bar

  • 43c4e98 Add ImagingFileExperiment

  • e823b52 base and dummy attenuatorbox added

  • ece80dd Add flat field correction task

  • 1ec03be addons: Enable async I/O by ImageWriter

  • 6535790 Add docs command to Concert program

  • 0f91277 Add Flask-like extension system

  • 6c02a84 abort on ctrl-c

  • align axis: auto-compute metric_eps by default (better rotation axis alignment)

Pull Requests

  • #374 from ufo-kit/calculate_motor_velocity

  • #389 from ufo-kit/add-flatcorrect

  • #396 from ufo-kit/fix-388

  • #419 from ufo-kit/add-percent-quantity

  • #436 from ufo-kit/align-rotation-axis

  • #450 from ufo-kit/experiment-state

  • #447 from ufo-kit/device-names

  • #445 from ufo-kit/experiment-logs

  • #437 from ufo-kit/add-lens-selector

  • #442 from ufo-kit/always-execute-finish-in-experiment

  • #443 from ufo-kit/add_enable_disable_to_motors

  • #434 from ufo-kit/add-state-to-experiment

  • #435 from ufo-kit/add-external-softlimits

  • #439 from ufo-kit/add-state-to-monochromator

  • #339 from ufo-kit/ufo-multi-gpus

  • #432 from ufo-kit/fix-requirements-python2.7

  • #433 from ufo-kit/fix-typo-in-experiment

  • #430 from ufo-kit/cleanup-writers

  • #429 from ufo-kit/add-readers

  • #425 from ufo-kit/add_parameters_to_experiments

  • #428 from ufo-kit/dependabot/pip/pyxdg-0.26

  • #427 from ufo-kit/fix-fsm-transitioning-error

  • #417 from ufo-kit/add-progressbar

  • #412 from ufo-kit/enable-writer-offset

  • #411 from ufo-kit/multipage-file-camera

  • #410 from ufo-kit/faster-dummy-camera

  • #408 from ufo-kit/write_multi

  • #405 from ufo-kit/add-attenuator

  • #401 from ufo-kit/fix-400

  • #399 from ufo-kit/Flake8

  • #397 from ufo-kit/fix-367

  • #395 from ufo-kit/fix-385

  • #394 from ufo-kit/docs

  • #393 from ufo-kit/docs

  • #391 from ufo-kit/forbid-addon-reattachment

  • #387 from ufo-kit/extend-queue

  • #379 from ufo-kit/fix-flatcorrectedbackproject

  • #383 from ufo-kit/fix-dimax-start-recording

  • #381 from ufo-kit/fix-ufo-segfault

  • #370 from ufo-kit/generalize-acquisitions

  • #378 from ufo-kit/print-noasync-traceback

  • #377 from ufo-kit/accumulate-addon

  • #376 from ufo-kit/numpy-accumulate

  • #375 from ufo-kit/fix-DirectoryWalker-relative-paths

  • #369 from ufo-kit/extensions

Fixes

  • #400: Fix properties mixup of uca-cameras

  • #392 and support for current Pint versions’

  • #388: Fixed units checking of Numeric

  • #385: Optional output of resolve without units

  • #371: override grab only in uca cameras

  • #301: don’t print traceback for DISABLE_ASYNC

Version 0.10

Released on February 9th 2015.

Improvements

  • Uca cameras support optional parameters.

  • We added convenience functions for acquiring certain image types like dark fields, etc.

  • We can determine tomographic rotation axis based on the convenince functions mentioned above.

  • Hard-limit state is allowed as target in motor’s home method.

  • Added a decorator for measuring function execution time.

  • Added XRayTube device.

  • Added Gripper device.

  • Added asynchronous grab Camera.grab_async.

  • Added SampleChanger device.

  • Parameter setters are abortable. Thanks to that we added the abort function to stop actions on devices. It can be used also per-device.

  • Simplified concert.base, we don’t use metaclasses anymore.

  • Added normalize for intensity normalization.

  • Added Camera.convert for converting images before they are returned by the camera’s grab method (useful for flipping, etc.).

  • Added a generic process coroutine which takes a callable and applies it to the coming data.

  • We check soft limits for correct unit.

  • Added EDF reading support via fabio.

  • Added experiment Addon which operate on the data produced by an experiment (e.g. image viewing, online reconstruction, etc.).

  • Added n-dimensional scans.

  • Added ssh+tmux support via concert-server and concert-connect.

  • Added session export command.

  • Added session loading via –filename.

  • Walker can write data stored in lists, not only in a coroutine way.

API breaks

  • Renamed fetch command to import.

  • Use positive config names (ENABLE_ instead of DISABLE_).

Fixes

  • Various beam time fixes from #345.

  • IPython version check in #332.

  • #300, #301, #306, #308, #310, #331, #353.

Version 0.9

Released on August 15th 2014.

Improvements

  • The state machine mechanism is not special anymore but directly inherits from Parameter.

  • Added walker mechanism to write sequence data in hierarchical structures such as directories or HDF5 files.

  • The long-standing gevent integration with IPython is finished at least for IPython >= 2.0.

  • Added @expects decorator to annotate what a function can receive.

  • Added async.resolve() to get result of future lists.

  • Added accumulate sink and timer coroutines.

  • Added Timestamp class for PCO cameras that decodes the BCD timestamp embedded in a frame.

  • Added optional wait_on to getter and setter of a ParameterValue.

  • We now raise an exception in if a uca frame is not available.

  • Experiments have now hooks for preparation and cleanup tasks.

  • Added basic control loop classes.

  • Add binary signal device class.

API breaks

  • scan yields futures instead of returning a list

  • Moved specific pco cameras to concert.devices.cameras.pco.

  • Moved write_images to concert.storage

  • Removed base.MultiContext and base.Process

Fixes

  • #198, #254, #271, #277, #280, #286, #293

  • The pint dependency had to be raised to 0.5.2 in order to compute sums of quantities.

Version 0.8

Released on April 16th 2014.

Improvements

  • concert log can now --follow the current operation.

  • Soft limits and parameters can be locked both temporarily and permanently.

  • Added new @quantity decorator for simple cases.

  • The concert` binary can now be started without a session.

  • Added cross-correlation tomographic axis finding.

  • Added frame consumer to align_rotation_axis.

  • Simplify file camera and allow resetting it

  • Added ports property to the base IO device.

  • Added Photodiode base device class.

  • Added Fiber-Lite halogen lightsource.

  • Added LEDs connected within the wago.

  • Added stream coroutine to cameras.

  • Added EdmundOptics photodiode.

  • Added PCO.4000 camera.

  • Added Wago input/output device.

API breaks

  • Raise CameraError instead of ValueError

  • Change Pco’s freerun to stream

Fixes

  • Fix FileCamera pixel units in grab

  • Import GLib.GError correctly

  • Make recording context exception-safe

  • Fix quantity problem with recent Pint versions

  • #200, #203, #206, #209, #228, #230, #245

Version 0.7

Released on February 17th 2014.

Improvements

  • Added beam finding and centering

  • threaded decorator uses daemonic threads

  • Added downsize, queue, stall, PickSlice to coroutine filters

  • Added reconstruction of the whole volume using UFO Framework

  • Documentation was restructured significantly (split to usage/API)

  • Added tomography helper functions

  • Crio motor support continuous rotation

  • PyplotViewer can be configured for faster drawing capabilities using blit

  • Added dummy Scales

  • Tests cover all devices (at least try to instantiate them)

  • Added pixel units, q.pixel (shorthand q.px)

  • Changed prompt color to terminal default

  • Added Positioner device

  • Added Detector device

API Breaks

  • Finite state machine was reworked significantly

  • Motors were cleaned from mixins and hard-limit was incorporated into them

  • recording() context was added to cameras

  • backprojector coroutine filter was significantly simplified

  • average_images arguments changed

  • Experiments were completely restructured based on usage of Acquisition

  • PyplotViewer plotting signature changed

  • Remove leftover beam line specific shutters

  • Many getters/setters were replaced by properties, especially in the concert.ext.viewers module

  • Appropriate get_ set_ functions were replaced by non-prefixed ones

Fixes

  • #118, #128, #132, #133, #139, #148, #149, #150, #157, #159, #165, #169, #173, #174, #175, #176, #178, #179, #181, #184, #189, #192

Version 0.6

Released on December 10th 2013.

Improvements

  • Concert now comes with an experimental gevent backend that will eventually replace the thread pool executor based asynchronous infrastructure.

  • Each device can now have an explicit State object and @transition applied to function which will change the state depending on the successful outcome of the decorated function.

  • 1D data plotting is implemented as PyplotCurveViewer.

  • The concert binary now knows the cp command to make a copy of a session. The start command can receive a log level and with the --non-interactive option run a session as a script.

  • Devices and parameters can store their current parameter values with stash and restore them later with restore.

  • Changed the IPython prompt.

  • Added the NewPort 74000 Monochromator.

  • Provide a require function that will scream when the required Concert version is not installed.

API breaks

  • Motor is renamed to LinearMotor for all devices.

  • Parameter objects are now declared at class-level instead of at run-time within the class constructor.

  • concert.storage.create_folder renamed to concert.storage.create_directory

  • concert.ext.viewers.PyplotViewer substituted by 1D and 2D viewers concert.ext.viewers.PyplotCurveViewer and concert.ext.viewers.PyplotImageViewer

  • To wait on a Future you have to call .join instead of .wait.

  • Coroutine functions and decorators moved to concert.coroutines[.base], asynchronous functions and decorators moved to concert.async.

  • Removed is_async

  • Configuration moved to concert.config

  • Method names of concert.ext.ufo.InjectProcess changed.

Fixes

  • #168, #166, #152, #147, #158, #150, #157, #95, #138

  • Many more concerning the camera implementation.

Version 0.5

Released on October 31st 2013.

Improvements

  • Python 3 is supported and can be tested with tox.

  • Most imports are delayed in the concert binary to reduce startup time.

  • We do not depend on Logbook anymore but use Python’s logging module.

  • Experiments can now be modelled with the concert.experiments module.

  • concert.ext.viewers.PyplotViewer can be used to show 2D image data.

  • Spyder command plugin is now available. That means if you have Spyder installed you can control Concert from an IDE instead of from IPython.

  • Tests were restructured for easier access.

API breaks

  • concert.connections package moved to concert.networking module

  • Renamed concert.helpers.multicast to broadcast to reflect its true purpose.

  • Session helpers such as dstate and ddoc have been moved to concert.session.utils.

  • Frames grabbed with the libuca devices will return a copy instead of the same buffer.

Fixes:

  • #106, #113 and many more which did not deserve an issue number.

Version 0.4

Released on October 7th 2013.

Improvements

  • Tests and rotation axis alignment is faster now.

  • Soft limits were added to the parameter (accessible with .lower and .upper)

  • Cleaner inet connection implemention.

  • Base pumps and scales were added.

  • Concert no longer depends on testfixtures for running tests.

  • Started work on flexible data processing schemes for light computation based on a coroutine approach.

  • Integrated tifffile.py in case libtiff is not available.

  • concert mv renames sessions.

  • @threaded decorator can be used to run a function in its own thread.

  • Scanner parameters can now be set in the constructor.

  • Parameters can now be locked independently of the parent device. However, if done so, no one else can lock the device.

  • Add code_of function to show the source of a function.

  • Introduced coroutine based data processing facility.

API breaks

  • Renamed to_steps to to_device and do not drop units

  • camera.grab returns None if no data is available

  • uca.Camera exposes the wrapped GObject camera as an attribute called uca instead of camera.

  • minimum, maximum and intervals are now longer implemented as Parameter objects of Scanner but simple attributes.

  • asynchronous module content has been moved to helpers

  • Removed Scanner class in favor of scan function.

Fixes:

  • Integration with all IPython releases works again.

  • runtests.py returns 0 on success.

  • #19, #55, #71, #78, #79

Version 0.3

Released on August 19th 2013.

Note: This release breaks Python 2.6 compatibility!

  • Calibration classes moved to concert.devices.calibration

  • Remove concert.processes.focus and reorganize concert.optimization package, the focusing can be implemented by Maximizer with a proper feedback.

  • Add --repo parameter to the fetch command. With this flag, session files version controlled with Git can be imported.

  • Use pint instead of quantities. pint is faster for smaller Numpy arrays, stricter and does not depend on Numpy.

  • Things can now run serialized if concert.asynchronous.DISABLE is set to True.

  • Restructured tests into separate directories.

  • Fix PDF generation of the docs.

  • Fix problem with IPython version >= 0.10.

Version 0.2

Released on July 14th 2013.

  • Move third-party code to concert.ext. For example get_tomo_scan_result must be imported from concert.ext.nexus.

  • Adds concert fetch to pull session files from remote locations.

  • Code cleanup

Version 0.1.1

Bug fix release, released on May 25th 2013

  • Fixes Python 3 support.

  • Monochromator fix.

Version 0.1

First public release.