Add AMD SEV-SNP vm support #3682

zhencliu · 2023-05-10T06:04:36Z

ID: 783

bssrikanth · 2023-11-21T08:39:33Z

Hello @zhencliu , I wanted to give this a try. Is it a recent version which is pushed here in this draft or do you have any recent updates which you are yet to push into this draft which resolves conflicts with recent master?

zhencliu · 2023-11-21T09:02:24Z

Hello @zhencliu , I wanted to give this a try. Is it a recent version which is pushed here in this draft or do you have any recent updates which you are yet to push into this draft which resolves conflicts with recent master?

Hi, thanks for looking into this, this patch was not updated since I pushed it, it's out of date, the reason is SNP is not yet supported even in upstream, so we still don't have a final qemu command line which can be running a SNP VM, the options/params are TBD, talked with VT maintainers, we'll wait till the qemu upstream is ready.

Fix SNP mem allocation issue ontop of SNP draft PR avocado-framework#3682 Signed-off-by: Srikanth Aithal <[email protected]>

Fix SNP mem allocation issue ontop of SNP draft PR avocado-framework#3682 + AMDSEV fix Signed-off-by: Srikanth Aithal <[email protected]>

zixi-chen · 2024-10-17T07:01:01Z

@zhencliu after #3928 merged, does this patch need to update?

qemu cmdline sample: -object sev-snp-guest,id=sev0, \ cbitpos=51,reduced-phys-bits=1, \ policy=0x30000, \ id-block=YWFhYWFhYWFhYWFhYWFhCg==, \ id-auth=CxHK/OKLkXGn/KpAC7Wl1FSiisWDbGTEKz..., \ auth-key-enabled=on, \ host-data=LNkCWBRC5CcdGXirbNUV1OrsR28s..., \ guest-visible-workarounds=AA==, \ Signed-off-by: Zhenchao Liu <[email protected]>

zhencliu · 2024-10-18T01:46:41Z

@zhencliu after #3928 merged, does this patch need to update?

Hi @zixi-chen , conflict resolved, thanks.

bssrikanth · 2024-10-23T11:18:20Z

@zhencliu I was testing this PR, when SNP guest boots I also see memory-backend-ram getting assigned by default in addition to the memory-backend-memfd object. When further debugged I see this place

avocado-vt/virttest/qemu_devices/qcontainer.py

Line 3469 in 3a65355

params.setdefault("backend", "memory-backend-ram")

assigns a memory-backend-ram object by default. IMO if users passes memory-backend-memfd we can skip adding this default memory-backend-ram object. Please let me know your thoughts.

zhencliu · 2024-10-24T02:40:39Z

@zhencliu I was testing this PR,

Thanks.

when SNP guest boots I also see memory-backend-ram getting assigned by default in addition to the memory-backend-memfd object. When further debugged I see this place

avocado-vt/virttest/qemu_devices/qcontainer.py

Line 3469 in 3a65355

params.setdefault("backend", "memory-backend-ram")

assigns a memory-backend-ram object by default.

The default memory-backend-ram is used only when we don't assign any value

IMO if users passes memory-backend-memfd we can skip adding this default memory- backend-ram object. Please let me know your thoughts.

Yes, may I ask how you pass memory-backend-memfd ? Actually we have a cartesian param "vm_mem_backend" to be used for assign the memory backend, you can set it as
vm_mem_backend = memory-backend-memfd
and then have a try, then memory-backend-ram will not be used

bssrikanth · 2024-10-24T04:59:19Z

@zhencliu I was testing this PR,

Thanks.

when SNP guest boots I also see memory-backend-ram getting assigned by default in addition to the memory-backend-memfd object. When further debugged I see this place

avocado-vt/virttest/qemu_devices/qcontainer.py

Line 3469 in 3a65355

params.setdefault("backend", "memory-backend-ram")

assigns a memory-backend-ram object by default.

The default memory-backend-ram is used only when we don't assign any value

IMO if users passes memory-backend-memfd we can skip adding this default memory- backend-ram object. Please let me know your thoughts.

Yes, may I ask how you pass memory-backend-memfd ? Actually we have a cartesian param "vm_mem_backend" to be used for assign the memory backend, you can set it as vm_mem_backend = memory-backend-memfd and then have a try, then memory-backend-ram will not be used

Thanks for getting back.

I was using backend_mem=memory-backend-memfd, now I ran with vm_mem_backend = memory-backend-memfd -> it did not make any difference I continue to see memory-backend-ram during vm run.

I am listing configs I use from a memory and vm type below, please let me know your config:

use_mem = no
vm_secure_guest_type = snp
vm_sev_cbitpos = 51
vm_sev_reduced_phys_bits = 1
vm_mem_backend = memory-backend-memfd
mem_devs = "upm0"
vm_sev_upm_devs = "upm0"
vm_sev_upm_memdev_upm0 = upm0
size_mem_upm0 = 4096M

zhencliu · 2024-10-24T06:37:47Z

@zhencliu I was testing this PR,

Thanks.

when SNP guest boots I also see memory-backend-ram getting assigned by default in addition to the memory-backend-memfd object. When further debugged I see this place

avocado-vt/virttest/qemu_devices/qcontainer.py

Line 3469 in 3a65355

params.setdefault("backend", "memory-backend-ram")

assigns a memory-backend-ram object by default.

The default memory-backend-ram is used only when we don't assign any value

IMO if users passes memory-backend-memfd we can skip adding this default memory- backend-ram object. Please let me know your thoughts.

Yes, may I ask how you pass memory-backend-memfd ? Actually we have a cartesian param "vm_mem_backend" to be used for assign the memory backend, you can set it as vm_mem_backend = memory-backend-memfd and then have a try, then memory-backend-ram will not be used

Thanks for getting back.

I was using backend_mem=memory-backend-memfd, now I ran with vm_mem_backend = memory-backend-memfd -> it did not make any difference I continue to see memory-backend-ram during vm run.

I am listing configs I use from a memory and vm type below, please let me know your config:
use_mem = no
vm_secure_guest_type = snp
vm_sev_cbitpos = 51
vm_sev_reduced_phys_bits = 1
vm_mem_backend = memory-backend-memfd
mem_devs = "upm0"
vm_sev_upm_devs = "upm0"
vm_sev_upm_memdev_upm0 = upm0
size_mem_upm0 = 4096M

I see, vm_mem_backend is used for system memory, i.e. you will see the object id in machine option (e.g. -machine q35, xxx, memory-backend=mem_backend_obj_id). You defined the mem_devs, but set use_mem = no, which means no memory device will be generated.

I cannot figure out what happened to your code, would you paste your code change, or the qemu command line in your debug log?

bssrikanth · 2024-10-24T07:31:28Z

@zhencliu I am testing out this patch on top of avocado-vt master, there is no custom code which I have added.

Common config:

vm_secure_guest_type = snp
mem_devs = "upm0"
backend_mem = "memory-backend-memfd"
vm_mem_backend = memory-backend-memfd
share_upm0 = true
prealloc_upm0 = false
reserve_upm0 = false
vm_sev_upm_devs = "upm0"
vm_sev_upm_memdev_upm0 = upm0
size_mem_upm0 = 4096M

Using above common config without use_mem = no hit issue mentioned below:

[stdlog] MALLOC_PERTURB_=1  /home/VT_BUILD/usr/local/bin/qemu-system-x86_64 \
[stdlog]     -machine q35,confidential-guest-support=lsec0,memory-backend=mem-machine_mem \
[stdlog]     -object sev-snp-guest,id=lsec0,cbitpos=51,reduced-phys-bits=1,kernel-hashes=off  \
[stdlog]     -m 1024 \
[stdlog]     -object memory-backend-memfd,size=4096M,prealloc=false,share=true,reserve=false,id=mem-upm0 \
[stdlog]     -device pc-dimm,id=dimm-upm0,memdev=mem-upm0 \
[stdlog]     -object memory-backend-memfd,size=1024M,id=mem-machine_mem  \
..
[stdlog] 2024-10-24 07:15:05,695 avocado.virttest.qemu_vm client           L0635 INFO | [qemu output] qemu-system-x86_64: -device pc-dimm,id=dimm-upm0,memdev=mem-upm0: no slots where allocated, please specify the 'slots' option

Using common config with use_mem = no I see the SNP boot test passing, below are the mem related commandline generated:

[stdlog] MALLOC_PERTURB_=1  /home/VT_BUILD/usr/local/bin/qemu-system-x86_64 \
[stdlog]     -machine q35,confidential-guest-support=lsec0,memory-backend=mem-machine_mem \
[stdlog]     -object sev-snp-guest,id=lsec0,cbitpos=51,reduced-phys-bits=1,kernel-hashes=off  \
[stdlog]     -m 1024 \
[stdlog]     -object memory-backend-memfd,size=4096M,prealloc=false,share=true,reserve=false,id=mem-upm0 \
[stdlog]     -object memory-backend-memfd,size=1024M,id=mem-machine_mem  \
..

zhencliu · 2024-10-24T07:57:29Z

@zhencliu I am testing out this patch on top of avocado-vt master, there is no custom code which I have added.

Common config:

vm_secure_guest_type = snp
mem_devs = "upm0"
backend_mem = "memory-backend-memfd"
vm_mem_backend = memory-backend-memfd
share_upm0 = true
prealloc_upm0 = false
reserve_upm0 = false
vm_sev_upm_devs = "upm0"
vm_sev_upm_memdev_upm0 = upm0
size_mem_upm0 = 4096M

Using above common config without use_mem = no hit issue mentioned below:

[stdlog] MALLOC_PERTURB_=1  /home/VT_BUILD/usr/local/bin/qemu-system-x86_64 \
[stdlog]     -machine q35,confidential-guest-support=lsec0,memory-backend=mem-machine_mem \
[stdlog]     -object sev-snp-guest,id=lsec0,cbitpos=51,reduced-phys-bits=1,kernel-hashes=off  \
[stdlog]     -m 1024 \
[stdlog]     -object memory-backend-memfd,size=4096M,prealloc=false,share=true,reserve=false,id=mem-upm0 \
[stdlog]     -device pc-dimm,id=dimm-upm0,memdev=mem-upm0 \
[stdlog]     -object memory-backend-memfd,size=1024M,id=mem-machine_mem  \
..
[stdlog] 2024-10-24 07:15:05,695 avocado.virttest.qemu_vm client           L0635 INFO | [qemu output] qemu-system-x86_64: -device pc-dimm,id=dimm-upm0,memdev=mem-upm0: no slots where allocated, please specify the 'slots' option

Use slots_mem param to set the slots

Using common config with use_mem = no I see the SNP boot test passing, below are the mem related commandline generated:
[stdlog] MALLOC_PERTURB_=1 /home/VT_BUILD/usr/local/bin/qemu-system-x86_64
[stdlog] -machine q35,confidential-guest-support=lsec0,memory-backend=mem-machine_mem
[stdlog] -object sev-snp-guest,id=lsec0,cbitpos=51,reduced-phys-bits=1,kernel-hashes=off
[stdlog] -m 1024
[stdlog] -object memory-backend-memfd,size=4096M,prealloc=false,share=true,reserve=false,id=mem-upm0
[stdlog] -object memory-backend-memfd,size=1024M,id=mem-machine_mem
..

So what is upm0 used for? Can we remove it? I mean the following configuration is OK for memory

vm_secure_guest_type = snp
vm_sev_cbitpos = 51
vm_sev_reduced_phys_bits = 1
vm_mem_backend = memory-backend-memfd

Would you give your expected qemu command line for memory device? Then we can show you some sample configurations, currently we are confused for your usage of memory

bssrikanth · 2024-10-25T04:29:21Z

Would you give your expected qemu command line for memory device? Then we can show you some sample configurations, currently we are confused for your usage of memory

@zhencliu there were some confusions, now its clear. I have tested SNP boot with memfd successfully with this patch. I am now in process of trying out different memory configurations. I will let you know here how it goes. Thank you for your time till now!

bssrikanth · 2024-11-04T05:13:40Z

@zhencliu I could able to test this patch out in my environment with different memory topologies.
Tested-by: Srikanth Aithal [email protected]

zixi-chen · 2024-11-14T07:25:21Z

@zhencliu The patch LGTM, we have enabled the test loop, could we merge this one?

zhencliu · 2024-11-14T07:42:15Z

hi @luckyh , would you please have a look ? Thanks.

zhencliu force-pushed the snp branch 4 times, most recently from 12fe5e9 to 0ca6462 Compare May 11, 2023 02:04

bssrikanth pushed a commit to bssrikanth/avocado-vt that referenced this pull request Nov 30, 2023

Fix SNP issue

da75595

Fix SNP mem allocation issue ontop of SNP draft PR avocado-framework#3682 Signed-off-by: Srikanth Aithal <[email protected]>

bssrikanth pushed a commit to bssrikanth/avocado-vt that referenced this pull request Nov 30, 2023

Make SNP work

a8f1bda

Fix SNP mem allocation issue ontop of SNP draft PR avocado-framework#3682 + AMDSEV fix Signed-off-by: Srikanth Aithal <[email protected]>

zhencliu force-pushed the snp branch 2 times, most recently from bdb8008 to 77d17f1 Compare June 7, 2024 10:22

zhencliu force-pushed the snp branch from 77d17f1 to 5ab7994 Compare August 14, 2024 07:25

zhencliu marked this pull request as ready for review August 14, 2024 07:26

zhencliu force-pushed the snp branch 2 times, most recently from 32af24f to 749b0a5 Compare August 15, 2024 03:28

zhencliu force-pushed the snp branch from 749b0a5 to 4a77195 Compare October 17, 2024 07:43

luckyh approved these changes Nov 14, 2024

View reviewed changes

luckyh merged commit 90271e5 into avocado-framework:master Nov 14, 2024
50 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add AMD SEV-SNP vm support #3682

Add AMD SEV-SNP vm support #3682

zhencliu commented May 10, 2023 •

edited

Loading

bssrikanth commented Nov 21, 2023 •

edited

Loading

zhencliu commented Nov 21, 2023

zixi-chen commented Oct 17, 2024

zhencliu commented Oct 18, 2024

bssrikanth commented Oct 23, 2024

zhencliu commented Oct 24, 2024 •

edited

Loading

bssrikanth commented Oct 24, 2024

zhencliu commented Oct 24, 2024 •

edited

Loading

bssrikanth commented Oct 24, 2024

zhencliu commented Oct 24, 2024

bssrikanth commented Oct 25, 2024 •

edited

Loading

bssrikanth commented Nov 4, 2024

zixi-chen commented Nov 14, 2024

zhencliu commented Nov 14, 2024

Add AMD SEV-SNP vm support #3682

Add AMD SEV-SNP vm support #3682

Conversation

zhencliu commented May 10, 2023 • edited Loading

bssrikanth commented Nov 21, 2023 • edited Loading

zhencliu commented Nov 21, 2023

zixi-chen commented Oct 17, 2024

zhencliu commented Oct 18, 2024

bssrikanth commented Oct 23, 2024

zhencliu commented Oct 24, 2024 • edited Loading

bssrikanth commented Oct 24, 2024

zhencliu commented Oct 24, 2024 • edited Loading

bssrikanth commented Oct 24, 2024

zhencliu commented Oct 24, 2024

bssrikanth commented Oct 25, 2024 • edited Loading

bssrikanth commented Nov 4, 2024

zixi-chen commented Nov 14, 2024

zhencliu commented Nov 14, 2024

zhencliu commented May 10, 2023 •

edited

Loading

bssrikanth commented Nov 21, 2023 •

edited

Loading

zhencliu commented Oct 24, 2024 •

edited

Loading

zhencliu commented Oct 24, 2024 •

edited

Loading

bssrikanth commented Oct 25, 2024 •

edited

Loading