concatLatestFrom included in RxJS

[hadrien-toma]

NgRx introduced concatLatestFrom in v11, it seems to be interesting in certain performances contexts so I would like to know if it could/should be included in RxJS directly and if no, for which considerations it should be out of scope.

Does someone know if concatLatestFrom could/should be introduced in RxJS? 🙂

[karptonite]

Hey, I'm a bit late to this, but I agree that this would be a good addition to rxjs. I use it now in ngrx, and prior to that, I used my own home-brewed version of that operator. I imagine it would be useful beyond ngrx.

For a better explanation of what this does, see concatLatestFrom here.

But, in short (and cribbing from the link above), it works link withLatestFrom, but lazily evaluates the provided Observable factory. This allows you to use the source value when selecting additional sources to concat, and prevents evaluation until necessary.

[voliva]

but then isn't it the same as withLatestFrom(defer(() => observable$))?

[karptonite]

It is not. The difference is that you can use the source value to generate the observable whose value you are retrieving.

[kwonoj]

The actual implementation is nothing more than a combination of existing operators: https://github.com/ngrx/platform/blob/ee00ed404386b8be252dd095b3f3a8a6f5b8c90f/modules/effects/src/concat_latest_from.ts#L55-L57

I do not believe rxjs core need to expand its api surface to include.

[karptonite]

Isn't that effectively true of many of the combined operators (switchMap, mergeMap, etc)? It may not be the way they are implemented now, but, for example, the description of mergeMap is "Maps each value to an Observable, then flattens all of these inner Observables using mergeAll."

The API expands its surface to accommodate such useful combinations of operators as a convenience; later, they may be implemented in a more efficient way. It may be that concatLatestFrom is not useful enough to justify addition, but I don't think that merely being currently implemented as a combination of existing operators (in almost 20 lines of code) is a good argument not to add this.

[kwonoj]

Isn't that effectively true of many of the combined operators (switchMap, mergeMap, etc)?

Yes, we're in the slow process of removing those from core. The only reason we don't do it a single day is it's huge major breaking changes for a lot of people.

There's really no strong reason a certain simple composable operator need to be in a core api surface; our intention is core provide necessary pieces for any other userland operators and encourage to publish userland operators to opt in for the additional utilities.

[karptonite]

I can certainly see the argument for removing those from the core. But it seems that if they are removed, they should be removed to some other official rxjs library; while you can make switchMap, for example, with a simple combination of operators, that isn't the way it is done in rxjs, presumably because there are subtle improvements (perhaps in performance?) involved in doing it another way. Having one official place to get well-performing operators would seem to be important. If such a library were added, I think that that would be a good place for concatLatestFrom.

[kwonoj]

But it seems that if they are removed, they should be removed to some other official rxjs library

Yes again, that's the reason (again) core team takes careful approach to remove / deprecate from core. It is mostly legacy, historical reason we included in core but that doesn't justifies to remove at once: so we are taking cautious approach for reducing api surface.

Adding new operator for the same crux, we won't add new operator unless we are absolutely certain 1. it is so widely used (even if it's simple composition operator) that worth to core rxjs maintains 2. absolutely complex or handles internals of observables that userland operator cannot fulfill. For the cases of 1, we strongly encourage to publish it as userland npm pkg to gauge those interest: otherwise, there's no way to tell if it's actually being widely used or not since rxjs does not have way to telemetry user's api usages.

The core principle of rxjs core is balancing off between providing sufficient api surface vs. not overload api surface for the users, especially beginners and usual users would not need a lot of operators. While we can't present as numbers (again we don't have telemetry), in most cases users of rxjs won't need full api surface of core as of today.

[karptonite]

I see that this has been closed, which seems odd, since it is just a discussion.

To motivate this just a bit more: I've use rxjs pretty extensively in an Angular application since before Angular 2 was released. In all the time I used it, only once was I motivated to write my own operator. I called it withFirstFrom, and I used it for years until ngrx released concatLatestFrom, which did almost exactly what my withFirstFrom did, but better.

The point is that this operator was really the only gap I ever found for my needs in the rxjs operators. Obviously an anecdote is not data, but I do think that this operator fills a valuable missing role in the official API.

[kwonoj]

answered in here #6974 (reply in thread)

specifically

For the cases of 1, we strongly encourage to publish it as userland npm pkg to gauge those interest: otherwise, there's no way to tell if it's actually being widely used or not since rxjs does not have way to telemetry user's api usages.

We don't accept expanding api surfaces from an immediate request since that doesn't reflect the actual usage.

Closing discussion does not mean we'll not reconsider this ever. We'll constantly evaluate surroundings if there are sufficient backing numbers.

[voliva]

@karptonite I was confused at your answer that you get the value at your function. Reading through the implementation of concatLatestFrom looks like an operator extremely coupled to the kind of observables that ngrx uses: The observables that you return from that function must emit something synchronously, or your values will get dropped.

In this sandbox you can see this effect: https://codesandbox.io/p/sandbox/gallant-water-qv775h?file=%2Fsrc%2Findex.html%3A10%2C7

// OK
timer(0, 1000)
  .pipe(
    take(3),
    concatLatestFrom((value) => of(`concatted-${value}`))
  )
  .subscribe((r) => console.log('OK', r));

// Not OK
const anotherObservable$ = timer(0).pipe(map(() => 'some async value'));

timer(0, 1000)
  .pipe(
    take(3),
    concatLatestFrom(() => anotherObservable$)
  )
  .subscribe((r) => console.log('Not OK', r));

// withLatestFrom actually works
timer(0, 1000)
  .pipe(
    take(3),
    withLatestFrom(anotherObservable$)
  )
  .subscribe((r) => console.log('withLatestFrom', r));

I think the place where you would usually work with this kind of observables are in ngrx store's select. Outside of this I think it can be a bit more dangerous than anything else (or at least it should be very well documented that this won't work with any observable that doesn't emit synchronously.

[karptonite]

Very interesting! But is it necessary that it emit synchronously, or just necessary that it emit after the source observable (or are we saying the same thing?)?

My own implementation of similar functionality, FYI, was:

import { Observable, first, map, mergeMap } from 'rxjs';

// this operator takes a function that returns an observable.
// it passes values from the source observable, and combines them,
// in the style of withLatestFrom, with the first value emitted from the
// newly created observable.
export const withFirstFrom =
  <T, R>(fn: (input: T) => Observable<R>) =>
  (source: Observable<T>) =>
    source.pipe(
      mergeMap(input =>
        fn(input).pipe(
          first(),
          map<R, [T, R]>(result => [input, result]),
        ),
      ),
    );