Use Pluto to control physical systems

[fonsp]

We are very interested to learn how Pluto can be used to control physical systems, and we would like to hear your ideas on how Pluto can be improved to better support this use case!

Why?

Controlling physical systems is important, but the software to do so is either difficult to use (e.g. C++), or incredibly easy to use (e.g. micro:bit) but not designed for 'adults'. Being able to control physical systems directly from Pluto would mean that you can work in the same environment as your analysis and modeling.

If you have experience in this subject, try out Pluto and tell us your ideas! We might have money for hardware and there might be options for a GSOC-like project, get in touch! [email protected]

[oschulz]

I'm actually considering Pluto for such applications (control of lab hardware and controlling/monitoring data acquisition. Pretty much our whole lab is Julia-based now, so it's the logical next step for us. Haven't done any real tests yet, but I'll gladly share feedback.

[fonsp]

That's awesome! Do let us know how it goes! @pankgeorg and I would be happy to video call and look at it together!

[oschulz]

Love to - hopefully we'll be able to ramp up our lab activities come summer (when Covid restrictions will hopefully relax).

[notinaboat]

Hi @fonsp,
I'm using Julia to build test automation systems.
The product being tested has various sensors, hydraulic actuators, pumps, interconnected control boards etc.
The test harness uses r-pi boards running Julia to do data acquisition, simulated sensor inputs, fault simulation etc.
I've been using Pluto to document test runs: Notes, plots, discussion, sub-plots of interesting filtered detail etc.
I have considered using Pluto for more interactive control of the test rig, but I haven't got a clear Idea of the use-case yet.

An issue that comes to mind is the interaction between Pluto's re-evaluation of cells and code that has real-world side-effects. I haven't learned enough about how Pluto does re-evaluation to know it would be a problem, but it would be important to for Pluto to not unexpectedly re-run code that changes the physical state of the system.

[fonsp]

That sounds super interesting, I would love to get your feedback!

the interaction between Pluto's re-evaluation of cells and code that has real-world side-effects

Exactly! This is what I would like to investigate more, the first thing that comes to mind is that a new UI concept like #298 could also be used to protect cells with real-world effects.

[Arryk]

Hi @fonsp
I've started using Julia switching from Matlab for control systems purpose. I've been experimenting with using Pluto to log interactivity a serial based devices using Plots.jl & LibSerialPort.jl

The idea is to very quickly setup and connect to a real-time OBC or µcontroller using Pluto, with quick data logging & data processing.

My first step was to use Pluto as a real-time charting system, taking advantage of the reactive nature of the notebook. I manage to create a very simple data logging system, however I cannot get the reactivity I would need to set up a real-time chart. (I can get the reactivity I want manually by spamming shift+enter but didn't find a way of doing programmatically)

Having the possibility to trigger reactivity from a @async statement would be something needed for this type of application, where a background routine collect & process data while triggering reactive blocks in the note book to plot/analysis. What is remarkable is in term of efficiency, reading some serial, decoding and plotting is almost as easy as doing it with LabVIEW, however I feel like the notebook is still lacking key features to control reactivity.

Here's an example of the code:

begin
# Serial
	sp = open("COM3", 100_000)
	set_read_timeout(sp, 2)
	set_write_timeout(sp, 2)
# Variable Init
	const I_SP_INDEX = 2
	const I_CV_INDEX = 7
	t = DateTime[]
	i_sp = Float64[]
	i_cv = Float64[]
end

@bind serial_state Radio(["Stop" => "Stop", "Run" => "Run"], default="Stop")

begin
	if serial_state == "Run"
		# sp_flush(sp, SP_BUF_BOTH)
		write(sp, "start\r\n") # Start Control Algorithm
		@async begin
			println("Capturing Data...")
			elapsed = 0
			x = empty!(t)
			y = empty!(i_cv)
			yr = empty!(i_sp)
			while serial_state == "Run"
				try
					serial_data = String(readline(sp))
					try
						sdname, sdval = parse_serial_data(serial_data)
						try
							y = push!(i_cv, sdval[I_CV_INDEX])
							yr = push!(i_sp, sdval[I_SP_INDEX])
							x = push!(t, now())
						catch
							println("Allocation Error!")
						end
					catch
						println("Parsing Error!")
					end
				catch
					println("Reading Error!")
				end
				sleep(0.1)
				elapsed += 0.1
				println("$elapsed s")
			end
			println("Done...")
		end
	else
		write(sp, "stop\r\n") # Stop Control
		if !isempty(t)
			dt = broadcast(-,t, t[1])./Millisecond(1000)
		else
			dt = []
		end
		x = dt
		y = i_cv
		yr = i_sp
	end
end

I've been working as a senior control systems engineer on lots of real physical platform (drones, satellites, robots...) with LabVIEW, Matlab, C, C++ & now Julia. I'll be more than happy to help regarding those problematics

[oschulz]

After giving some thought, I think Pluto is missing an important feature for allowing it to control physical system safely is the concept of reactive indicators.

I think Observables would be ideal for that. Does Pluto support them already?

[Arryk]

After giving some thought, I think Pluto is missing an important feature for allowing it to control physical system safely is the concept of reactive indicators.

I think Observables would be ideal for that. Does Pluto support them already?

I did couple of test using observables, and it does work with Pluto, however it doesn't work with Plots. It definitely have its importance using async functions. I don't think the concept of proper reactive indicator can be emulated easily without diving into details on how Pluto handle its own reactive controls.

An other solution, for my specific problem of live graphing, not the problem related to controlling physical systems as a whole, is to allow push!(x) to trigger reactivity which is not the case atm

[oschulz]

Ah, darn. Plots-compatible Observable support in Pluto would be so useful for use cases like this. I've been using this in Jupyter to "real-time" plot incoming data, etc.

[Arryk]

Ah, darn. Plots-compatible Observable support in Pluto would be so useful for use cases like this. I've been using this in Jupyter to "real-time" plot incoming data, etc.

That's interesting, could you share a MWE of the code you use with Jupyter? I'll be interesting to test it. Thanks

[oschulz]

Sure - I'll have to dig it out, it was a proof-of-concept, then we changed some other code, will have to update it.

[stillyslalom]

I've written several lab instrumentation measurement/control systems in Labview, and my (positive!) experience with Pluto for data analysis would make me very hesitant to use it to control physical systems. There's a lot of promise here, and I'm excited by the idea, but also skeptical given the inflexibility of the physical world, and the high costs imposed (up to & including mortal danger) when a physical system goes astray.

The reactive programming model works well at the scale of a small notebook, but as the complexity of any notebook grows, the unseen connections between cell dependencies become harder to untangle. Physical systems tend to have the same sort of interdependent behavior, and the more complex the system, the harder it is to control those interactions. Trying to manage feedback loops between reactive software and "reactive" (and possibly dangerous) physical systems could quickly turn into a nightmare. The best way to limit the scope of these interdependencies within Pluto would be allowing sub-notebooks with clearly-defined inputs & outputs (and certainly no implicit global state).

The main challenge arises from the need to acquire/process sensor & control data without triggering unintentional reactivity. As an example, let's walk through the entire data-acquisition pipeline for the single most important part of any control system: a physical emergency stop button. The voltage across the E-stop button is read by some data-acquisition hardware (alongside several other simultaneous signal acquisitions for other sensors) at some regular interval (the "polling rate"). The data-acquisition hardware ("DAQ") communicates with the computer using some serial interface, and from there to software via an instrument driver. From there, Julia could either read the E-stop signal from the driver every time it is sent, drop the signal if it's busy with other tasks, or feed into a (circular/fixed-size/dynamic?) buffer. The corresponding variable (e_stop::Bool) is updated every time the signal is acquired, but the rest of the system isn't in a well-defined state when that signal arrives. Let's say there's some Pluto cell with an if-else case that handles e_stop == true, but there's another ifelse case that triggers some costly operation in another cell. If the e_stop condition isn't also handled in that downstream cell, that whole computation might be inadvertently triggered during the emergency-stop process.

On reflection, most of the issues that might arise when controlling a physical system with a reactive notebook could be handled by:

• encapsulation of user interface, data acquisition, instrument control, and signal processing into asynchronous sub-notebooks
• fine-grained control of reactivity, namely: being able to trigger a variable's dependent cells if and only if the variable is assigned in the active branch of a conditional

It'd also be nice to have 2D layouting and built-in (or at least fonsp-blessed) tools for prescribing the layout & appearance of buttons/switches/indicators/etc.

[oschulz]

high costs imposed (up to & including mortal danger) when a physical system goes astray

I agree - I tried out the new Pluto feature that allows for disabling cells, but I don't think that will work for such applications:

To control physical cells, we do IMHO need a Pluto "one-shot" cell mode - so that that cell must be explicitly triggered, and then runs only once. Also, we do want to see the output of all dependent cells, so graying them out (like the current "disable-cell" mode does) doesn't work well either. Ideally, when a "one-shot" cell is triggered, all dependent cells are run (except for other "one-shot" cells), and their output stays fully visible. Only when the input of the "one-shot" changes, that cell and all dependent cells would be greyed out, to indicate that their results are outdated.

[oschulz]

To control physical cells, we do IMHO need a Pluto "one-shot" cell mode - so that that cell must be explicitly triggered, and then runs only once.

Here's a more detailed suggestion in this direction, that (hopefully) is in keeping with Pluto philosophy:

#2574