Source Generator for Web Service

[robheffo79]

Hi,

I am developing a source generator for a project that takes data pulled from a Rest API and builds the appropriate code to match said data.

Using a Console Application I was able to successfully test my source generator by mocking up the ISourceGenerator GeneratorExecutionContext and GeneratorInitializationContext classes, and ensured the Source Generator is working as expected.

When I use the source generator in my build I am unable to make the required Rest API calls via HttpClient.

I have found reference to HttpClient not being allowed due to security policy. What is the point of developing and opening up this API for use by all developers if we cannot use it the way we need to?

Is there any way to be able to allow calls via HttpClient or work around this limitation in some way?

Regards,
Robert Heffernan

[svick]

I have found reference to HttpClient not being allowed due to security policy. What is the point of developing and opening up this API for use by all developers if we cannot use it the way we need to?

Can you explain what exactly do you mean? What is the full error message you're getting?

I can't think of anything specific to Roslyn that would prevent you from generating code that uses HttpClient (assuming the code would be able to use HttpClient itself without source generators).

[robheffo79]

In this case it's the source generator itself making the HttpClient request as part of the Execute method inside of my ISourceGenerator implementation, and not the code being produced.

Basically the flow is "Start Build->Invoke Source Generator->Make HTTP Get Request->Get Result->Convert Result Into Source Code->Add Source To Context->Finish Build"

[svick]

That still sounds like something specific to the way you build your code.

[robheffo79]

Debugging this is a nightmare. I managed to go from having no error messages in my Source Generator to coaxing out a message about it not being able to find a Nuget Dependency (System.Net.Http.Json). It seems that the code generator can't find the dependency when it's invoked by Roslyn. It only took a pile of Clean/Build cycles to get that to come out. So now I am chasing down that rabbit hole

[VolodymyrBS]

take a look at this issue 50148 .
adding dependencies to generators and Analyzers are a little bit tricky

[PathogenDavid]

Debugging this is a nightmare.

If you're trying to test the source generator within Visual Studio, consider testing on the command line instead. Visual Studio caches source generators too aggressively right now for iterative development.

[CyrusNajmabadi]

When I use the source generator in my build I am unable to make the required Rest API calls via HttpClient.

I would strongly recommend not doing this. This will have an extremely negative effect on the VS ide. You will effectively be doing unbounded network IO in a codepath that needs to run in sub-ms time max. Every time completoin comes up, this will be causing blocking network requests to happen, which will certainly impact hte ability for anyone using this generator to do basic stuff: like type.

Tagging @jasonmalinowski @chsienki for visibility and so we can provide appropriate docs steering people away from doing this.

@jasonmalinowski it will likely be very necessary for us to collect metrics on misbehaving generators so we can let people know why their performance is potentially tanking.

[robheffo79]

For me, this isn't a problem as the code being generated does nothing but inject a static constructor into a partial class. So there is nothing for the IDE to need to try and reflect. And from what I have seen of how aggressive VS caches the output from the Source Generator, I can't see it really being that much of a problem. The project this source generator is for is internal use, not public release. It will basically wind up having it's builds automated, only invoked when the dataset changes, causing an updated nuget package to be created and pushed to my nuget repository. I also have a static variable with the dataset stored in a class, and it gets fetched during the Initialize method ONLY if the static dataset isn't present. So my typing experience hasn't suffered at all given the fetch only ever happens once during the lifetime of the Source Generator's process. A few extra seconds during the compilation process doesn't matter either, I am using the Source Generator because it's portable and integrated into the compile process. Now if I can only just get this dependency problem to go away. I think that would be a better place to focus some time rather than writing some documents to say don't do unbound network IO. In fact given that it is actually quite reasonable to expect some Source Generators may need to do some lengthy operations during their initialization phase, having a way to do this asynchronously would be helpful.

…

On Mon, 18 Jan 2021, 10:43 CyrusNajmabadi, ***@***.***> wrote: When I use the source generator in my build I am unable to make the required Rest API calls via HttpClient. I would strongly recommend *not* doing this. This will have an extremely negative effect on the VS ide. You will effectively be doing unbounded *network* IO in a codepath that needs to run in sub-ms time *max*. Every time completoin comes up, this will be causing blocking network requests to happen, which will certainly impact hte ability for anyone using this generator to do basic stuff: like type. Tagging @jasonmalinowski <https://github.com/jasonmalinowski> @chsienki <https://github.com/chsienki> for visibility and so we can provide appropriate docs steering people away from doing this. @jasonmalinowski <https://github.com/jasonmalinowski> it will likely be *very* necessary for us to collect metrics on misbehaving generators so we can let people know why their performance is potentially tanking. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50568 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJAS34TOL3PYBPTMHAT2MKDS2NYZJANCNFSM4WF4MILA> .

[CyrusNajmabadi]

So there is nothing for the IDE to need to try and reflect.

The ide doesn't know this.

And from what I have seen of how aggressive VS caches t

Any caching is currently a bug.

[robheffo79]

So there is nothing for the IDE to need to try and reflect.

The ide doesn't know this.
I can see how the source generator providing a hint to the IDE that this is the case would be helpful.

And from what I have seen of how aggressive VS caches

Any caching is currently a bug.
Again, I can see how this would be helpful based on a hint from the source generator.

I could see it working something like this:

`
using Microsoft.CodeAnalysis;
using System;

namespace MySourceGenerator
{
[Generator(DynamicOutput: true, Cachable: true)]
public class MyGenerator : ISourceGenerator
{
public void Execute(GeneratorExecutionContext context)
{
// Generate Code Here!
}

    public void Initialize(GeneratorInitializationContext context)
    {
        // Do Initialization Here!
    }
}

}
`

Where the DynamicOutput argument provides a hint to the IDE that the output from the Source Generator constantly changes, and Cachable tells Roslyn that the output can be cached between builds, Deleted on a Clear build or Deleted and Regenerated on a Rebuild build.