Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test features in ci #802

Open
wants to merge 105 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
ffac148
WIP: Test features in ci
autoantwort Nov 9, 2022
adc9011
Add command line to specify which ports to test and how features are …
autoantwort Nov 9, 2022
5f99418
Use old clang-format
autoantwort Nov 9, 2022
80a53a3
optimize
autoantwort Nov 9, 2022
6946d6a
Don't delete packages folders
autoantwort Nov 10, 2022
a93c707
Don't purge packages folder to make the binary cache happy
autoantwort Nov 10, 2022
bbdf7e7
Clear internal binary cache state instead of keep packages folder bec…
autoantwort Nov 10, 2022
c6ed7b3
Don't try to build known failures.
autoantwort Nov 10, 2022
8760f5c
Merge branch 'main' into feature/ci-test-features
autoantwort Nov 12, 2022
a1a0188
Rename command line arguments
autoantwort Nov 12, 2022
1f9ebad
Make pipeline happy
autoantwort Nov 12, 2022
fcefd69
Fix windows
autoantwort Nov 12, 2022
e760b17
Merge branch 'main' into feature/ci-test-features
autoantwort Nov 19, 2022
4aeaf49
Fix spelling
autoantwort Nov 19, 2022
ab8591d
Merge branch 'main' into feature/ci-test-features
autoantwort Nov 30, 2022
517e614
Add support for a baseline file
autoantwort Dec 8, 2022
a610bb4
Make msvc happy
autoantwort Dec 8, 2022
1c208e8
Merge branch 'main' into feature/ci-test-features
autoantwort Dec 9, 2022
781c7b1
Create issue template for failing ports
autoantwort Dec 9, 2022
1d60a7d
Give the logs folder a better name
autoantwort Dec 10, 2022
be0dbda
Merge branch 'main' into feature/ci-test-features
autoantwort Dec 22, 2022
9a0c844
Collect time needed to build a port
autoantwort Dec 26, 2022
03eb18c
Don't include cascaded features in the "all" feature test
autoantwort Jan 15, 2023
e801de1
Fix warning
autoantwort Jan 15, 2023
f83b661
Fix msvc warnings
autoantwort Jan 15, 2023
625213b
Merge branch 'main' into feature/ci-test-features
autoantwort Jan 15, 2023
d49840f
Merge branch 'main' into feature/ci-test-features
autoantwort Jan 23, 2023
498b36b
Add option to only run feature tests
autoantwort Jan 27, 2023
673c36c
Merge branch 'main' into feature/ci-test-features
autoantwort Feb 9, 2023
a3beb6e
Add command test-features
autoantwort Feb 14, 2023
ebfc504
Remove feature test code from ci command. Add option to exclude known…
autoantwort Feb 14, 2023
dcdef4a
Remove leftover setting
autoantwort Feb 14, 2023
cf527fa
Merge branch 'main' into feature/ci-test-features
autoantwort Feb 14, 2023
d2edb00
No print2 anymore
autoantwort Feb 14, 2023
c53a769
Merge branch 'main' into feature/ci-test-features
autoantwort Feb 20, 2023
c407d6d
Merge branch 'main' into feature/ci-test-features
autoantwort Feb 28, 2023
e399b2d
Fix build
autoantwort Feb 28, 2023
1f2a2ad
Merge branch 'main' into feature/ci-test-features
autoantwort Mar 16, 2023
1b2519a
Fix build
autoantwort Mar 16, 2023
4cdc14c
Make sure all dependencies are tested before a FullPackageSpec is tested
autoantwort Mar 20, 2023
c7f5598
Use stable sort to keep features of ports together
autoantwort Mar 20, 2023
db0364c
Don't test unsupported ports
autoantwort Mar 23, 2023
61a453f
Fix usage of wrong variable
autoantwort Mar 24, 2023
2772c23
Do a full comparison if the vectors are not sorted
autoantwort Mar 24, 2023
c67d769
Don't use moved from variable
autoantwort Mar 26, 2023
6ed1709
Merge branch 'main' into feature/ci-test-features
autoantwort Mar 27, 2023
d4b1bce
Merge branch 'main' into feature/ci-test-features
autoantwort Apr 4, 2023
7f89ea8
Fix msvc build
autoantwort Apr 4, 2023
f111e15
Move CiFeatureBaseline classes into own files since they don't share …
autoantwort Apr 4, 2023
d188944
Merge branch 'main' into feature/ci-test-features
autoantwort Apr 26, 2023
78cb67f
Merge branch 'main' into feature/ci-test-features
autoantwort May 2, 2023
935ffd9
core is not allowed in non fail baseline entries
autoantwort May 5, 2023
fc6c3d8
Add support for options
autoantwort May 5, 2023
fab6107
Add combination-fails and feature-fails so that failing features are …
autoantwort May 5, 2023
82b52d5
File conflicts are not always the fault of the port
autoantwort May 6, 2023
6dbedb6
Don't add the core feature twice
autoantwort May 7, 2023
424d729
Fix msvc warning
autoantwort May 7, 2023
59174dc
Merge branch 'main' into feature/ci-test-features
autoantwort May 21, 2023
170d279
Fetch binary cache before installation
autoantwort May 23, 2023
c6845cf
Use PortAbiCache to reduce time spend on computing abis
autoantwort May 23, 2023
600a6d7
Batch binary caching pre checks to improve speed.
autoantwort May 23, 2023
2793f04
BinaryCaching: Only reset state to available if package was restored
autoantwort May 30, 2023
e2fc16d
Merge branch 'main' into feature/ci-test-features
autoantwort May 31, 2023
e774fb7
Fix msvc warning
autoantwort May 31, 2023
c525294
Fix msvc warning
autoantwort May 31, 2023
66f0514
Merge branch 'main' into feature/ci-test-features
autoantwort May 31, 2023
6d6f8a9
Fix msvc warning
autoantwort Jun 6, 2023
92de08a
Localisation
autoantwort Jun 6, 2023
1dc9663
More localization
autoantwort Jun 6, 2023
3b21335
More localization
autoantwort Jun 12, 2023
f5e037e
Merge branch 'main' into feature/ci-test-features
autoantwort Jun 13, 2023
fe08c5e
Fix build
autoantwort Jun 13, 2023
d6f922b
If the first tested feature is an expected cascade the whole tested f…
autoantwort Jun 16, 2023
ff9a23e
Emit error is feature is specified as cascade and feature-fails
autoantwort Jun 16, 2023
6d46b6a
Trigger Build
autoantwort Jun 16, 2023
3616c9a
Add message map
autoantwort Jun 17, 2023
7de7b89
Remove unnecessary changes
autoantwort Jun 21, 2023
948a7ba
Actually detect errors while parsing + detect more errors.
autoantwort Jun 27, 2023
021d1c9
Merge branch 'main' into feature/ci-test-features
autoantwort Jun 27, 2023
2428299
Restore old CMakeVarProvider interface changed in #1109
autoantwort Jun 28, 2023
f003d33
More localization + more information in result text
autoantwort Jun 28, 2023
d8ca81d
Merge branch 'main' into feature/ci-test-features
autoantwort Jul 12, 2023
7dfc95d
Merge branch 'main' into feature/ci-test-features
autoantwort Aug 15, 2023
e208e4d
Fix warning
autoantwort Aug 15, 2023
d2e15d4
Merge branch 'main' into feature/ci-test-features
autoantwort Aug 24, 2023
3a08f95
Add command to format a ci.feature.baseline.txt file
autoantwort Aug 27, 2023
6062fb8
Merge branch 'main' into feature/ci-test-features
autoantwort Aug 27, 2023
4a2d427
windows: Allow writing to file that is read
autoantwort Aug 28, 2023
9495a2a
Merge branch 'feature/ci-test-features' of github.com:autoantwort/vcp…
autoantwort Aug 28, 2023
07725fb
Merge branch 'main' into feature/ci-test-features
autoantwort Aug 29, 2023
8b112b9
Revert "windows: Allow writing to file that is read"
autoantwort Aug 29, 2023
e8eb42b
Merge branch 'main' into feature/ci-test-features
autoantwort Sep 7, 2023
09d83b8
Merge branch 'main' into feature/ci-test-features
autoantwort Sep 9, 2023
3c048e6
Rename options and rename command to x-test-features
autoantwort Sep 16, 2023
67c698b
Merge branch 'main' into feature/ci-test-features
autoantwort Oct 11, 2023
2471c5b
Merge fun with branch 'main' into feature/ci-test-features
autoantwort Nov 12, 2023
9e13600
Merge branch 'main' into feature/ci-test-features
autoantwort Dec 13, 2023
0658071
Merge branch 'main' into feature/ci-test-features
autoantwort Apr 17, 2024
3ab6b83
Fix merge conflict
autoantwort Apr 17, 2024
ea7e63e
Message db
autoantwort Apr 17, 2024
7af8217
Lof of fun fixing merge conflicts while Merging branch 'main' into fe…
autoantwort May 22, 2024
291a9d3
Merge branch 'main' into feature/ci-test-features and resolve conflict
autoantwort Aug 30, 2024
3da56e8
Merge branch 'main' into feature/ci-test-features
autoantwort Nov 18, 2024
43cd7c2
Fix merge conflicts
autoantwort Nov 18, 2024
8c2055b
Fix warning
autoantwort Nov 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions azure-pipelines/e2e-assets/ci-feature-baseline/asci-range.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
Was sorted by the "Sort Lines Ascending" command in VSCode (# was excluded because it is a comment)
| 32 | 20 | 00100000
! | 33 | 21 | 00100001
" | 34 | 22 | 00100010
$ | 36 | 24 | 00100100
% | 37 | 25 | 00100101
& | 38 | 26 | 00100110
' | 39 | 27 | 00100111
( | 40 | 28 | 00101000
) | 41 | 29 | 00101001
* | 42 | 2A | 00101010
+ | 43 | 2B | 00101011
, | 44 | 2C | 00101100
- | 45 | 2D | 00101101
. | 46 | 2E | 00101110
/ | 47 | 2F | 00101111
0 | 48 | 30 | 00110000
1 | 49 | 31 | 00110001
2 | 50 | 32 | 00110010
3 | 51 | 33 | 00110011
4 | 52 | 34 | 00110100
5 | 53 | 35 | 00110101
6 | 54 | 36 | 00110110
7 | 55 | 37 | 00110111
8 | 56 | 38 | 00111000
9 | 57 | 39 | 00111001
: | 58 | 3A | 00111010
; | 59 | 3B | 00111011
< | 60 | 3C | 00111100
= | 61 | 3D | 00111101
> | 62 | 3E | 00111110
? | 63 | 3F | 00111111
@ | 64 | 40 | 01000000
A | 65 | 41 | 01000001
B | 66 | 42 | 01000010
C | 67 | 43 | 01000011
D | 68 | 44 | 01000100
E | 69 | 45 | 01000101
F | 70 | 46 | 01000110
G | 71 | 47 | 01000111
H | 72 | 48 | 01001000
I | 73 | 49 | 01001001
J | 74 | 4A | 01001010
K | 75 | 4B | 01001011
L | 76 | 4C | 01001100
M | 77 | 4D | 01001101
N | 78 | 4E | 01001110
O | 79 | 4F | 01001111
P | 80 | 50 | 01010000
Q | 81 | 51 | 01010001
R | 82 | 52 | 01010010
S | 83 | 53 | 01010011
T | 84 | 54 | 01010100
U | 85 | 55 | 01010101
V | 86 | 56 | 01010110
W | 87 | 57 | 01010111
X | 88 | 58 | 01011000
Y | 89 | 59 | 01011001
Z | 90 | 5A | 01011010
[ | 91 | 5B | 01011011
\ | 92 | 5C | 01011100
] | 93 | 5D | 01011101
^ | 94 | 5E | 01011110
_ | 95 | 5F | 01011111
` | 96 | 60 | 01100000
a | 97 | 61 | 01100001
b | 98 | 62 | 01100010
c | 99 | 63 | 01100011
d | 100 | 64 | 01100100
e | 101 | 65 | 01100101
f | 102 | 66 | 01100110
g | 103 | 67 | 01100111
h | 104 | 68 | 01101000
i | 105 | 69 | 01101001
j | 106 | 6A | 01101010
k | 107 | 6B | 01101011
l | 108 | 6C | 01101100
m | 109 | 6D | 01101101
n | 110 | 6E | 01101110
o | 111 | 6F | 01101111
p | 112 | 70 | 01110000
q | 113 | 71 | 01110001
r | 114 | 72 | 01110010
s | 115 | 73 | 01110011
t | 116 | 74 | 01110100
u | 117 | 75 | 01110101
v | 118 | 76 | 01110110
w | 119 | 77 | 01110111
x | 120 | 78 | 01111000
y | 121 | 79 | 01111001
z | 122 | 7A | 01111010
{ | 123 | 7B | 01111011
| | 124 | 7C | 01111100
} | 125 | 7D | 01111101
~ | 126 | 7E | 01111110
16 changes: 16 additions & 0 deletions azure-pipelines/e2e-assets/ci-feature-baseline/ci.baseline.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
not-sup-host-b:arm64-osx=fail
not-sup-host-b:x64-osx=fail
not-sup-host-b:x86-windows=fail
not-sup-host-b:x64-windows=fail
not-sup-host-b:arm64-windows=fail
not-sup-host-b:x64-linux=fail
not-sup-host-b:arm64-linux=fail


dep-on-feature-not-sup:arm64-osx=fail
dep-on-feature-not-sup:x64-osx=fail
dep-on-feature-not-sup:x86-windows=fail
dep-on-feature-not-sup:x64-windows=fail
dep-on-feature-not-sup:arm64-windows=fail
dep-on-feature-not-sup:x64-linux=fail
dep-on-feature-not-sup:arm64-linux=fail
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
Was sorted by the "Sort Lines Ascending" command in VSCode (# was excluded because it is a comment)
| 32 | 20 | 00100000
_ | 95 | 5F | 01011111
- | 45 | 2D | 00101101
, | 44 | 2C | 00101100
; | 59 | 3B | 00111011
: | 58 | 3A | 00111010
! | 33 | 21 | 00100001
? | 63 | 3F | 00111111
. | 46 | 2E | 00101110
' | 39 | 27 | 00100111
" | 34 | 22 | 00100010
( | 40 | 28 | 00101000
) | 41 | 29 | 00101001
[ | 91 | 5B | 01011011
] | 93 | 5D | 01011101
{ | 123 | 7B | 01111011
} | 125 | 7D | 01111101
@ | 64 | 40 | 01000000
* | 42 | 2A | 00101010
/ | 47 | 2F | 00101111
\ | 92 | 5C | 01011100
& | 38 | 26 | 00100110
% | 37 | 25 | 00100101
` | 96 | 60 | 01100000
^ | 94 | 5E | 01011110
+ | 43 | 2B | 00101011
< | 60 | 3C | 00111100
= | 61 | 3D | 00111101
> | 62 | 3E | 00111110
| | 124 | 7C | 01111100
~ | 126 | 7E | 01111110
$ | 36 | 24 | 00100100
0 | 48 | 30 | 00110000
1 | 49 | 31 | 00110001
2 | 50 | 32 | 00110010
3 | 51 | 33 | 00110011
4 | 52 | 34 | 00110100
5 | 53 | 35 | 00110101
6 | 54 | 36 | 00110110
7 | 55 | 37 | 00110111
8 | 56 | 38 | 00111000
9 | 57 | 39 | 00111001
A | 65 | 41 | 01000001
a | 97 | 61 | 01100001
B | 66 | 42 | 01000010
b | 98 | 62 | 01100010
C | 67 | 43 | 01000011
c | 99 | 63 | 01100011
d | 100 | 64 | 01100100
D | 68 | 44 | 01000100
e | 101 | 65 | 01100101
E | 69 | 45 | 01000101
f | 102 | 66 | 01100110
F | 70 | 46 | 01000110
g | 103 | 67 | 01100111
G | 71 | 47 | 01000111
h | 104 | 68 | 01101000
H | 72 | 48 | 01001000
i | 105 | 69 | 01101001
I | 73 | 49 | 01001001
j | 106 | 6A | 01101010
J | 74 | 4A | 01001010
k | 107 | 6B | 01101011
K | 75 | 4B | 01001011
l | 108 | 6C | 01101100
L | 76 | 4C | 01001100
m | 109 | 6D | 01101101
M | 77 | 4D | 01001101
n | 110 | 6E | 01101110
N | 78 | 4E | 01001110
o | 111 | 6F | 01101111
O | 79 | 4F | 01001111
p | 112 | 70 | 01110000
P | 80 | 50 | 01010000
q | 113 | 71 | 01110001
Q | 81 | 51 | 01010001
r | 114 | 72 | 01110010
R | 82 | 52 | 01010010
s | 115 | 73 | 01110011
S | 83 | 53 | 01010011
t | 116 | 74 | 01110100
T | 84 | 54 | 01010100
u | 117 | 75 | 01110101
U | 85 | 55 | 01010101
v | 118 | 76 | 01110110
V | 86 | 56 | 01010110
w | 119 | 77 | 01110111
W | 87 | 57 | 01010111
x | 120 | 78 | 01111000
X | 88 | 58 | 01011000
y | 121 | 79 | 01111001
Y | 89 | 59 | 01011001
z | 122 | 7A | 01111010
Z | 90 | 5A | 01011010
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
not-sup-host-b:arm64-linux=fail
not-sup-host-b:arm64-osx=fail
not-sup-host-b:arm64-windows=fail
not-sup-host-b:x64-linux=fail
not-sup-host-b:x64-osx=fail
not-sup-host-b:x64-windows=fail
not-sup-host-b:x86-windows=fail


dep-on-feature-not-sup:arm64-linux=fail
dep-on-feature-not-sup:arm64-osx=fail
dep-on-feature-not-sup:arm64-windows=fail
dep-on-feature-not-sup:x64-linux=fail
dep-on-feature-not-sup:x64-osx=fail
dep-on-feature-not-sup:x64-windows=fail
dep-on-feature-not-sup:x86-windows=fail
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# first line
# second line

a = fail
b = fail
c = fail


# new block

a = cascade
b = cascade
c = cascade
13 changes: 13 additions & 0 deletions azure-pipelines/e2e-assets/ci-feature-baseline/keep-blocks.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# first line
# second line

b = fail
c = fail
a = fail


# new block

b = cascade
c = cascade
a = cascade
19 changes: 19 additions & 0 deletions azure-pipelines/end-to-end-tests-dir/format-feature-baseline.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
. $PSScriptRoot/../end-to-end-tests-prelude.ps1

$ciFeatureBaselines = (Get-Item "$PSScriptRoot/../e2e-assets/ci-feature-baseline").FullName
$testProjects = Get-ChildItem "$ciFeatureBaselines/*.txt" -File
$testProjects | % {
$asItem = Get-Item $_
$full = $asItem.FullName
$name = $asItem.Name
$expectedPath = "$ciFeatureBaselines/expected/$name"
$tempItemPath = "$TestingRoot/$name"
Write-Trace "test that format-feature-baseline on $full produces $expectedPath"
[string]$expected = Get-Content $expectedPath -Raw
Copy-Item $asItem $tempItemPath
Run-Vcpkg format-feature-baseline $tempItemPath
$actual = Get-Content $tempItemPath -Raw
if ($expected -ne $actual) {
throw "Expected formatting $full to produce $expectedPath but was $tempItemPath"
}
}
8 changes: 5 additions & 3 deletions include/vcpkg-test/mockcmakevarprovider.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@ namespace vcpkg::Test
dep_info_vars.emplace(spec, SMap{});
}

void load_tag_vars(const ActionPlan& action_plan, Triplet host_triplet) const override
void load_tag_vars(Span<const FullPackageSpec> specs,
Span<Path> /*port_locations*/,
Triplet host_triplet) const override
{
for (auto&& install_action : action_plan.install_actions)
tag_vars.emplace(install_action.spec, SMap{});
for (auto&& spec : specs)
tag_vars.emplace(spec.package_spec, SMap{});
(void)(host_triplet);
}

Expand Down
1 change: 1 addition & 0 deletions include/vcpkg/base/contractual-constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ namespace vcpkg
inline constexpr StringLiteral SwitchInstallRoot = "install-root";
inline constexpr StringLiteral SwitchJson = "json";
inline constexpr StringLiteral SwitchKeepGoing = "keep-going";
inline constexpr StringLiteral SwitchKnownFailuresFrom = "known-failures-from";
inline constexpr StringLiteral SwitchLinux = "linux";
inline constexpr StringLiteral SwitchManifestRoot = "manifest-root";
inline constexpr StringLiteral SwitchMaxRecurse = "max-recurse";
Expand Down
Loading
Loading