feat: add config sources

[Kyle-Verhoog]

Add support for setting and storing configuration from additional sources. This enables a setting to have values coming from a default, environment or programmatic source.

The default priority of the sources is (lowest to highest):

1. Default
2. Environment
3. Programmatic

[P403n1x87]

I love the general idea of keeping track of how a value was set. I just wonder if we could make use of descriptors and dynamic classes here to tidy the logic a bit. For example:

from enum import Enum


class Source(Enum):
    DEFAULT = 0
    OVERRIDE = 1


class VarInstance:
    def __init__(self, *args, **kwargs):
        try:
            super().__init__(*args, **kwargs)
        except Exception:
            pass
        self.source = Source.DEFAULT

    # Add support for internal mutation
    def append(self, *args, **kwargs):
        self.source = Source.OVERRIDE
        return super().append(*args, **kwargs)


def make_var_instance(_type):
    return type(f"VarInstance[{_type.__name__}]", (VarInstance, _type), {})


# These should be built on demand
IntVarInstance = make_var_instance(int)
ListVarInstance = make_var_instance(list)

# Descriptor wrapper to keep track of the source
class Var:
    def __init__(self, value):
        self._value = value
        self._source = None

    def __get__(self, obj, objtype=None):
        return self._value

    def __set__(self, obj, value):
        self._value = IntVarInstance(value)
        self._value.source = Source.OVERRIDE


class Config:
    var = Var(IntVarInstance(10))
    lst = Var(ListVarInstance([1, 2, 3]))


config = Config()
assert config.var == 10, config.var
assert config.var.source == Source.DEFAULT
assert config.lst == [1, 2, 3]
assert config.lst.source == Source.DEFAULT

config.var = 20
config.lst.append(4)
assert config.var == 20
assert config.var.source == Source.OVERRIDE
assert config.lst == [1, 2, 3, 4]
assert config.lst.source == Source.OVERRIDE

IIUC, one thing that this change doesn't handle is the case of internal mutation. That is, if a list is mutated programmatically, we don't set the source accordingly (should we?).

[Kyle-Verhoog]

@P403n1x87 is this something you would like to do right now or can we leave it for a future refactor? I prefer the simplicity at the moment since we're still fleshing out the feature

[P403n1x87]

Ah it would be hard for me to give an "objective" answer here, as TBH I think the descriptor + dynamic types approach is simpler and leaner in my head (and that's why I proposed it). So my honest answer is that I'd prefer we went with this other approach, if we can make it work. Perhaps you might want to get some more eyes on this PR from e.g. the guild so that you could have more opinions besides my very polarised one 🙂 .

[P403n1x87]

Can't we make this an Enum?

[Kyle-Verhoog]

Sure

[Kyle-Verhoog]

Oh, I opted for not since we need to support older Pythons still, is this a blocker @P403n1x87?

[P403n1x87]

🤔 I think I'm getting a bit confused about the aim again. If this is for something that needs to go in ddtrace, it would probably be a new feature, hence it will be first released in a 2.x release. There, we no longer have support for older Pythons. We also don't plan any new 1.x releases. Are we planning to use envier in other projects that still support Python 2?

[P403n1x87]

Suggested change

	"programmatic",
	"override",

?

[Kyle-Verhoog]

yeah I wasn't a fan of programmatic, override is better 👍

[Kyle-Verhoog]

With remote configuration being a thing I opted for code to reflect in-code changes to the configuration rather than override

[P403n1x87]

I believe this is being addressed elsewhere at the moment so we can close this for now.