Skip to content
This repository has been archived by the owner on Oct 22, 2021. It is now read-only.

BPM memory limits need to be translated to kubernetes notation #977

Open
jandubois opened this issue Jun 9, 2020 · 1 comment
Open

BPM memory limits need to be translated to kubernetes notation #977

jandubois opened this issue Jun 9, 2020 · 1 comment
Labels
bug Something isn't working unscheduled

Comments

@jandubois
Copy link
Member

CF is treating all memory sizes as base-2 based:

// ToBytes parses a string formatted by ByteSize as bytes. Note binary-prefixed and SI prefixed units both mean a base-2 units
// KB = K = KiB	= 1024
// MB = M = MiB = 1024 * K
// GB = G = GiB = 1024 * M
// TB = T = TiB = 1024 * G
// PB = P = PiB = 1024 * T
// EB = E = EiB = 1024 * P

from https://github.com/cloudfoundry/bytefmt/blob/cf55d52/bytes.go#L80-L86

The BPM spec says:

The memory limit to apply to this process. It is formatted as a number and then a single character for units e.g. 1G, 256M.

quarks copies these strings verbatim into the container spec, e.g. resources.limits.memory: 1G.

However, in k8s 1G means 10**9 and not 2**30, so is about 8% too low.

1G from the BPM spec needs to be translated into 1Gi to represent the same number to kubernetes.

$ kubectl explain pod.spec.containers.resources.limits.memory
KIND:     Pod
VERSION:  v1

FIELD:    memory <string>

DESCRIPTION:
     Quantity is a fixed-point representation of a number. It provides
     convenient marshaling/unmarshaling in JSON and YAML, in addition to
     String() and AsInt64() accessors. The serialization format is: <quantity>
     ::= <signedNumber><suffix> (Note that <suffix> may be empty, from the ""
     case in <decimalSI>.) <digit> ::= 0 | 1 | ... | 9 <digits> ::= <digit> |
     <digit><digits> <number> ::= <digits> | <digits>.<digits> | <digits>. |
     .<digits> <sign> ::= "+" | "-" <signedNumber> ::= <number> | <sign><number>
     <suffix> ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI> ::= Ki
     | Mi | Gi | Ti | Pi | Ei (International System of units; See:
     http://physics.nist.gov/cuu/Units/binary.html) <decimalSI> ::= m | "" | k |
     M | G | T | P | E (Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the
     capitalization.) <decimalExponent> ::= "e" <signedNumber> | "E"
     <signedNumber> No matter which of the three exponent forms is used, no
     quantity may represent a number greater than 2^63-1 in magnitude, nor may
     it have more than 3 decimal places. Numbers larger or more precise will be
     capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be
     extended in the future if we require larger or smaller quantities. When a
     Quantity is parsed from a string, it will remember the type of suffix it
     had, and will use the same type again when it is serialized. Before
     serializing, Quantity will be put in "canonical form". This means that
     Exponent/suffix will be adjusted up or down (with a corresponding increase
     or decrease in Mantissa) such that: a. No precision is lost b. No
     fractional digits will be emitted c. The exponent (or suffix) is as large
     as possible. The sign will be omitted unless the number is negative.
     Examples: 1.5 will be serialized as "1500m" 1.5Gi will be serialized as
     "1536Mi" Note that the quantity will NEVER be internally represented by a
     floating point number. That is the whole point of this exercise.
     Non-canonical values will still parse as long as they are well formed, but
     will be re-emitted in their canonical form. (So always use canonical form,
     or don't diff.) This format is intended to make it difficult to use these
     numbers without writing some sort of special handling code in the hopes
     that that will cause implementors to also use a fixed point implementation.
@jandubois jandubois added the bug Something isn't working label Jun 9, 2020
@cf-gitbot
Copy link

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/173255409

The labels on this github issue will be updated when the story is started.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working unscheduled
Projects
None yet
Development

No branches or pull requests

2 participants