Description
Seeing kernel crash errors on an M1 Mac with one of our images. Was working prevously, but not sure is the OS or docker has been upgraded since.
Reproduce
$ mkdir test && touch test/go.py
# cap SYS_ADMIN is required as the container creates read-write overlay mounts at runtime
$ docker run --rm -ti --cap-add=SYS_ADMIN -v "$(PWD)/test":/repo:ro public.ecr.aws/bbc-ats/bbc-ats-lint
Running linters
ERRO[0000] error waiting for container:
Expected behavior
Container should run OK, not crash the Linux VM
docker version
Client:
Cloud integration: v1.0.35+desktop.5
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:28:49 2023
OS/Arch: darwin/arm64
Context: desktop-linux
Server: Docker Desktop 4.24.0 (122432)
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:31:36 2023
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.6.22
GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca
runc:
Version: 1.1.8
GitCommit: v1.1.8-0-g82f18fe
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 24.0.6
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.5
Path: /Users/starkd06/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.22.0-desktop.2
Path: /Users/starkd06/.docker/cli-plugins/docker-compose
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: /Users/starkd06/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: /Users/starkd06/.docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.8
Path: /Users/starkd06/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/starkd06/.docker/cli-plugins/docker-sbom
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: /Users/starkd06/.docker/cli-plugins/docker-scan
scout: Docker Scout (Docker Inc.)
Version: v1.0.7
Path: /Users/starkd06/.docker/cli-plugins/docker-scout
Server:
Containers: 3
Running: 0
Paused: 0
Stopped: 3
Images: 2
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
runc version: v1.1.8-0-g82f18fe
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.4.16-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 11
Total Memory: 7.667GiB
Name: docker-desktop
ID: 99583941-1097-41a4-a9f8-0a59eb661c6d
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: daemon is not using the default seccomp profile
Diagnostics ID
F123AF07-8555-41C8-8B6A-13C59C0AA3FB/20231009144422
Additional Info
Full error is:
Fatal error reported: Linux kernel v6.4.16 crash on virtualization.framework
[ 214.238908] Unable to handle kernel NULL pointer dereference at virtual address 000000000000046c
[ 214.239039] Mem abort info:
[ 214.239058] ESR = 0x0000000096000006
[ 214.239097] EC = 0x25: DABT (current EL), IL = 32 bits
[ 214.239154] SET = 0, FnV = 0
[ 214.239209] EA = 0, S1PTW = 0
[ 214.239262] FSC = 0x06: level 2 translation fault
[ 214.239305] Data abort info:
[ 214.239345] ISV = 0, ISS = 0x00000006
[ 214.239389] CM = 0, WnR = 0
[ 214.239425] user pgtable: 4k pages, 48-bit VAs, pgdp=000000010a6ec000
[ 214.239551] [000000000000046c] pgd=0800000140ff4003, p4d=0800000140ff4003, pud=0800000154c05003, pmd=0000000000000000
[ 214.239638] Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
[ 214.239738] Modules linked in: xfrm_user xfrm_algo nfsd auth_rpcgss nfs lockd grace sunrpc fakeowner(O) shiftfs(O) grpcfuse(O) vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock
[ 214.239949] CPU: 6 PID: 2272 Comm: black Tainted: G O 6.4.16-linuxkit #1
[ 214.240036] pstate: 41401005 (nZcv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)
[ 214.240154] pc : errseq_sample+0x14/0x30
[ 214.240215] lr : do_dentry_open+0x60/0x38c
[ 214.240247] sp : ffff80000be5b8a0
[ 214.240309] x29: ffff80000be5b8a0 x28: 0000000000000000 x27: 0000000000000000
[ 214.240399] x26: 0000000000000004 x25: ffffbc1e1a36f3f8 x24: 0000000000000004
[ 214.240476] x23: 0000000000000000 x22: ffff31421968e510 x21: ffff314221803b00
[ 214.240561] x20: ffff314221803b00 x19: 000000000000046c x18: 0000000000000000
[ 214.240634] x17: 0000000000000000 x16: ffffbc1e1908d658 x15: 0000000000000000
[ 214.240698] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[ 214.240829] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffbc1e1909cf08
[ 214.240939] x8 : ffff31421968e600 x7 : 0000000000000000 x6 : 0000000000000064
[ 214.241035] x5 : 0000000000000000 x4 : 00000000ffffffff x3 : ffff314219648b00
[ 214.241126] x2 : 0000000100000000 x1 : 0000000000000000 x0 : 000000000000046c
[ 214.241188] Call trace:
[ 214.241222] errseq_sample+0x14/0x30
[ 214.241261] do_dentry_open+0x60/0x38c
[ 214.241317] open_with_fake_path+0x58/0x84
[ 214.241363] fakeowner_open_realfile+0x60/0x8c [fakeowner]
[ 214.241421] fakeowner_open+0x38/0x74 [fakeowner]
[ 214.241482] do_dentry_open+0x2f8/0x38c
[ 214.241507] open_with_fake_path+0x58/0x84
[ 214.241535] ovl_open_realfile+0xbc/0xe8
[ 214.241567] ovl_open+0x74/0xbc
[ 214.241613] do_dentry_open+0x2f8/0x38c
[ 214.241685] vfs_open+0x3c/0x4c
[ 214.241717] path_openat+0x9d8/0xb30
[ 214.241758] do_filp_open+0x74/0xe4
[ 214.241806] do_sys_openat2+0xa8/0x10c
[ 214.241875] do_sys_open+0x5c/0x8c
[ 214.241911] __arm64_sys_openat+0x30/0x40
[ 214.241956] invoke_syscall.constprop.0+0x88/0xd8
[ 214.242002] do_el0_svc+0xc4/0x128
[ 214.242037] el0_svc+0x9c/0xcc
[ 214.242084] el0t_64_sync_handler+0xac/0x13c
[ 214.242150] el0t_64_sync+0x190/0x194
[ 214.242191] Code: a9be7bfd 910003fd f9000bf3 aa0003f3 (b9400260)
[ 214.242249] ---[ end trace 0000000000000000 ]---
Description
Seeing kernel crash errors on an M1 Mac with one of our images. Was working prevously, but not sure is the OS or docker has been upgraded since.
Reproduce
$ mkdir test && touch test/go.py
# cap SYS_ADMIN is required as the container creates read-write overlay mounts at runtime
$ docker run --rm -ti --cap-add=SYS_ADMIN -v "$(PWD)/test":/repo:ro public.ecr.aws/bbc-ats/bbc-ats-lint
Running linters
ERRO[0000] error waiting for container:
Expected behavior
Container should run OK, not crash the Linux VM
docker version
Client: Cloud integration: v1.0.35+desktop.5 Version: 24.0.6 API version: 1.43 Go version: go1.20.7 Git commit: ed223bc Built: Mon Sep 4 12:28:49 2023 OS/Arch: darwin/arm64 Context: desktop-linux Server: Docker Desktop 4.24.0 (122432) Engine: Version: 24.0.6 API version: 1.43 (minimum version 1.12) Go version: go1.20.7 Git commit: 1a79695 Built: Mon Sep 4 12:31:36 2023 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.6.22 GitCommit: 8165feabfdfe38c65b599c4993d227328c231fca runc: Version: 1.1.8 GitCommit: v1.1.8-0-g82f18fe docker-init: Version: 0.19.0 GitCommit: de40ad0docker info
Diagnostics ID
F123AF07-8555-41C8-8B6A-13C59C0AA3FB/20231009144422
Additional Info
Full error is:
Fatal error reported: Linux kernel v6.4.16 crash on virtualization.framework
[ 214.238908] Unable to handle kernel NULL pointer dereference at virtual address 000000000000046c
[ 214.239039] Mem abort info:
[ 214.239058] ESR = 0x0000000096000006
[ 214.239097] EC = 0x25: DABT (current EL), IL = 32 bits
[ 214.239154] SET = 0, FnV = 0
[ 214.239209] EA = 0, S1PTW = 0
[ 214.239262] FSC = 0x06: level 2 translation fault
[ 214.239305] Data abort info:
[ 214.239345] ISV = 0, ISS = 0x00000006
[ 214.239389] CM = 0, WnR = 0
[ 214.239425] user pgtable: 4k pages, 48-bit VAs, pgdp=000000010a6ec000
[ 214.239551] [000000000000046c] pgd=0800000140ff4003, p4d=0800000140ff4003, pud=0800000154c05003, pmd=0000000000000000
[ 214.239638] Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
[ 214.239738] Modules linked in: xfrm_user xfrm_algo nfsd auth_rpcgss nfs lockd grace sunrpc fakeowner(O) shiftfs(O) grpcfuse(O) vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock
[ 214.239949] CPU: 6 PID: 2272 Comm: black Tainted: G O 6.4.16-linuxkit #1
[ 214.240036] pstate: 41401005 (nZcv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)
[ 214.240154] pc : errseq_sample+0x14/0x30
[ 214.240215] lr : do_dentry_open+0x60/0x38c
[ 214.240247] sp : ffff80000be5b8a0
[ 214.240309] x29: ffff80000be5b8a0 x28: 0000000000000000 x27: 0000000000000000
[ 214.240399] x26: 0000000000000004 x25: ffffbc1e1a36f3f8 x24: 0000000000000004
[ 214.240476] x23: 0000000000000000 x22: ffff31421968e510 x21: ffff314221803b00
[ 214.240561] x20: ffff314221803b00 x19: 000000000000046c x18: 0000000000000000
[ 214.240634] x17: 0000000000000000 x16: ffffbc1e1908d658 x15: 0000000000000000
[ 214.240698] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[ 214.240829] x11: 0000000000000000 x10: 0000000000000000 x9 : ffffbc1e1909cf08
[ 214.240939] x8 : ffff31421968e600 x7 : 0000000000000000 x6 : 0000000000000064
[ 214.241035] x5 : 0000000000000000 x4 : 00000000ffffffff x3 : ffff314219648b00
[ 214.241126] x2 : 0000000100000000 x1 : 0000000000000000 x0 : 000000000000046c
[ 214.241188] Call trace:
[ 214.241222] errseq_sample+0x14/0x30
[ 214.241261] do_dentry_open+0x60/0x38c
[ 214.241317] open_with_fake_path+0x58/0x84
[ 214.241363] fakeowner_open_realfile+0x60/0x8c [fakeowner]
[ 214.241421] fakeowner_open+0x38/0x74 [fakeowner]
[ 214.241482] do_dentry_open+0x2f8/0x38c
[ 214.241507] open_with_fake_path+0x58/0x84
[ 214.241535] ovl_open_realfile+0xbc/0xe8
[ 214.241567] ovl_open+0x74/0xbc
[ 214.241613] do_dentry_open+0x2f8/0x38c
[ 214.241685] vfs_open+0x3c/0x4c
[ 214.241717] path_openat+0x9d8/0xb30
[ 214.241758] do_filp_open+0x74/0xe4
[ 214.241806] do_sys_openat2+0xa8/0x10c
[ 214.241875] do_sys_open+0x5c/0x8c
[ 214.241911] __arm64_sys_openat+0x30/0x40
[ 214.241956] invoke_syscall.constprop.0+0x88/0xd8
[ 214.242002] do_el0_svc+0xc4/0x128
[ 214.242037] el0_svc+0x9c/0xcc
[ 214.242084] el0t_64_sync_handler+0xac/0x13c
[ 214.242150] el0t_64_sync+0x190/0x194
[ 214.242191] Code: a9be7bfd 910003fd f9000bf3 aa0003f3 (b9400260)
[ 214.242249] ---[ end trace 0000000000000000 ]---