Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking] Support transport types other than tcp #65

Open
2 of 8 tasks
TheButlah opened this issue Sep 23, 2020 · 8 comments
Open
2 of 8 tasks

[Tracking] Support transport types other than tcp #65

TheButlah opened this issue Sep 23, 2020 · 8 comments

Comments

@TheButlah
Copy link
Contributor

TheButlah commented Sep 23, 2020

We should probably aim to support more transport types than just TCP.

Essentials:

  • tcp://
  • inproc://
  • ipc://

UDP and Multicast:

  • epgm://
  • pgm://
  • udp://

WebSockets (see https://rfc.zeromq.org/spec/39/):

  • ws://
  • wss://
@TheButlah
Copy link
Contributor Author

TheButlah commented Sep 23, 2020

@Alexei-Kornienko Any thoughts on what I could do to help implement inproc and ipc?

@Alexei-Kornienko
Copy link
Collaborator

@TheButlah basically I planned to 1st complete full tcp support and then start working on something else.
However if you are interested in other transport types you can start with some kind of proof of concept.

Currently most of the tcp related stuff is concentrated in utils module - https://github.com/zeromq/zmq.rs/blob/master/src/util.rs#L144
You can start with refactoring of this part and making such functions generic and replacing TcpStream with Stream trait to make it work with any kind of stream.
If we can do that than we can wrap other types of streams into existing codec and rest of the code should work without changes

@TheButlah
Copy link
Contributor Author

As part of this effort, I've introduced #68 which adds the necessary representations to disambiguate between endpoint variants and their transports.

@Alexei-Kornienko
Copy link
Collaborator

Alexei-Kornienko commented Sep 28, 2020

IMHO we should consider some kind of priority list for all this transports.
For example I don't think that inproc transport is really important cause inproc mean some part of rust code talks to other part of rust code in the same process. IMHO rust has already pretty good means for that (channels, etc.) so I don't think that this transport type will be used much (unless I would actually start a project to write a message broker using rust and zeromq as base).
ipc transport is again not very useful cause it can only communicate on a single machine (and we now live in world of docker and kubernetes)

Other transport types seem interesting and should worth talking a look

@TheButlah
Copy link
Contributor Author

TheButlah commented Sep 28, 2020

I actually find inproc and ipc to be really helpful. I tend to use inproc for integration tests in my C++ code that uses ZMQ, as I don't have to worry about binding to ports that might conflict with other running processes. I also think both are really useful when not using docker (and can't you still bind mount unix sockets in docker?), as well as when writing code that uses both C++ and rust. For example, you can have both the C++ component and rust component talk over ipc or inproc, and rust channels don't work in C++.

Also, the inproc and ipc protocols are allegedly supposed to be faster than the tcp one, and sometimes it's helpful to continue to use the ZMQ sockets for inproc communication, just to design a system flexible enough to seamlessly switch to tcp when run on another machine (or container). We use the c++ version this way at my work.

That being said, my personal interest is communicating over tcp in a wasm environment and that's what most likely I would work on next, but I'm sure we could support all of these eventually, especially if any helpful contributors are out there looking for something to do 👀

@TheButlah
Copy link
Contributor Author

On the topic of whether IPC is redundant with TCP: https://stackoverflow.com/a/29436429

It says that even when using localhost and tcp, ipc still has much better latency and throughout

@TheButlah
Copy link
Contributor Author

You can start with refactoring of this part and making such functions generic and replacing TcpStream with Stream trait to make it work with any kind of stream.
If we can do that than we can wrap other types of streams into existing codec and rest of the code should work without changes

I've done this now in #77

@TheButlah
Copy link
Contributor Author

An IPC implementation is ready in #84

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants