LIME Is Mediated Emulation - Revision history

Arthur: /* Compatibility With Other ABIs */

2022-08-23T17:14:45Z

Compatibility With Other ABIs

← Older revision		Revision as of 17:14, 23 August 2022
Line 23:		Line 23:

	=== Compatibility With Other ABIs ===		=== Compatibility With Other ABIs ===
	To solve the problems of 'ABI lock-in' discussed in the "[https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ Why Computers Suck]" post it may be possible to port LIME support to multiple different guest operating systems allowing ~~seamless~~ execution of any ABI with near indistinguishable performance compared to native execution.		To solve the problems of 'ABI lock-in' discussed in the "[https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ Why Computers Suck]" post it may be possible to port LIME support to multiple different guest operating systems allowing execution of any program with any ABI with near indistinguishable performance compared to native execution.

	=== Quality of Service (QoS) ===		=== Quality of Service (QoS) ===

Arthur: /* Development Status */

2022-08-23T17:14:13Z

Development Status

← Older revision		Revision as of 17:14, 23 August 2022
Line 32:		Line 32:

	=== VirtIO-FS ===		=== VirtIO-FS ===
	VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in ~~issue~~ [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>724 [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).		VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>724 [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).

	While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as disk D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].		While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as disk D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].

Arthur at 13:28, 23 August 2022

2022-08-23T13:28:33Z

← Older revision		Revision as of 13:28, 23 August 2022
Line 6:		Line 6:
	Compared to WINE, and Proton the benefits of this approach are that Win64 binaries execute without the incompatibilities introduced in new games taking advantage of the latest version of [https://docs.microsoft.com/en-us/windows/win32/directx DirectX], or when changes are introduced to the latest version of the Win64 [https://infogalactic.com/info/Application_binary_interface ABI].		Compared to WINE, and Proton the benefits of this approach are that Win64 binaries execute without the incompatibilities introduced in new games taking advantage of the latest version of [https://docs.microsoft.com/en-us/windows/win32/directx DirectX], or when changes are introduced to the latest version of the Win64 [https://infogalactic.com/info/Application_binary_interface ABI].

	This functionality is enabled via the use of [https://docs.linux-gvm.org/ GPU Virtual Machine] (for consumer GPU Virtualization), [https://virtio-fs.gitlab.io/design.html VirtIO-FS], [https://looking-glass.io/ Looking Glass] (for host <-> guest shared frame buffer), and [https://github.com/Arc-Compute/LibVF.IO/tree/master// LibVF.IO] (for orchestration).		This functionality is enabled via the use of [https://docs.linux-gvm.org/ GPU Virtual Machine] (for consumer GPU Virtualization), [https://virtio-fs.gitlab.io/design.html VirtIO-FS] (for host <-> guest filesystem sharing), [https://looking-glass.io/ Looking Glass] (for host <-> guest shared frame buffer), and [https://github.com/Arc-Compute/LibVF.IO/tree/master// LibVF.IO] (for orchestration).

	== Use-Cases ==		== Use-Cases ==

Arthur at 13:28, 23 August 2022

2022-08-23T13:28:00Z

← Older revision		Revision as of 13:28, 23 August 2022
Line 24:		Line 24:
	=== Compatibility With Other ABIs ===		=== Compatibility With Other ABIs ===
	To solve the problems of 'ABI lock-in' discussed in the "[https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ Why Computers Suck]" post it may be possible to port LIME support to multiple different guest operating systems allowing seamless execution of any ABI with near indistinguishable performance compared to native execution.		To solve the problems of 'ABI lock-in' discussed in the "[https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ Why Computers Suck]" post it may be possible to port LIME support to multiple different guest operating systems allowing seamless execution of any ABI with near indistinguishable performance compared to native execution.

			=== Quality of Service (QoS) ===
			[https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core.git/tree/Documentation/driver-api/vfio-mediated-device.rst VFIO-Mdev] functionality like that used in [https://docs.linux-gvm.org/ GPU Virtual Machine (GVM)] provides granular GPU resource allocation and load balancing functions similar to those seen in [https://infogalactic.com/info/Cgroups control groups (cgroups)] as they are commonly applied today to RAM and CPU resources. In the context LIME VFIO-Mdev could be applied for load balancing of individual processes running in a virtualized state to ensure individual processes making use of the GPU cannot deny service to others on the system.

	== Development Status ==		== Development Status ==

Arthur at 13:21, 23 August 2022

2022-08-23T13:21:19Z

← Older revision		Revision as of 13:21, 23 August 2022
Line 20:		Line 20:

	=== Win64 Orchestration ===		=== Win64 Orchestration ===
	Because LIME necessarily provisions application data, program files, and [https://openmdev.io/index.php/Glossary#NUMA_Node NUMA]-aware vCPU/vGPU resources (in the context of multi-GPU, multi-socket CPU systems) through LibVF.IO for automatic execution by one or more VM guests LIME might provide an ideal architecture for orchestration of Win64 applications at scale. This could include execution of programs for game streaming in a server environment, or multiple applications in a single user environment for '[https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md#using-windows-appvms-in-seamless-mode seamless-mode]' execution of guest programs similar to that of [https://~~github~~.~~com/Qubes~~-~~Community/Contents/blob/master/docs/~~os/~~windows~~/~~windows-tools.md~~ QubesOS] with hardware GPU acceleration added.		Because LIME necessarily provisions application data, program files, and [https://openmdev.io/index.php/Glossary#NUMA_Node NUMA]-aware vCPU/vGPU resources (in the context of multi-GPU, multi-socket CPU systems) through LibVF.IO for automatic execution by one or more VM guests LIME might provide an ideal architecture for orchestration of Win64 applications at scale. This could include execution of programs for game streaming in a server environment, or multiple applications in a single user environment for '[https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md#using-windows-appvms-in-seamless-mode seamless-mode]' execution of guest programs similar to that of [https://www.qubes-os.org/doc/ QubesOS] with hardware GPU acceleration added.

	=== Compatibility With Other ABIs ===		=== Compatibility With Other ABIs ===

Arthur at 13:20, 23 August 2022

2022-08-23T13:20:50Z

← Older revision		Revision as of 13:20, 23 August 2022
Line 20:		Line 20:

	=== Win64 Orchestration ===		=== Win64 Orchestration ===
	Because LIME necessarily provisions application data, program files, and [https://openmdev.io/index.php/Glossary#NUMA_Node NUMA]-aware vCPU/vGPU resources (in the context of multi-GPU, multi-socket CPU systems) through LibVF.IO for automatic execution by one or more VM guests LIME might provide an ideal architecture for orchestration of Win64 applications at scale. This could include execution of programs for game streaming in a server environment, or multiple applications in a single user environment for seamless execution of guest programs similar to that of [https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md QubesOS] with hardware GPU acceleration added.		Because LIME necessarily provisions application data, program files, and [https://openmdev.io/index.php/Glossary#NUMA_Node NUMA]-aware vCPU/vGPU resources (in the context of multi-GPU, multi-socket CPU systems) through LibVF.IO for automatic execution by one or more VM guests LIME might provide an ideal architecture for orchestration of Win64 applications at scale. This could include execution of programs for game streaming in a server environment, or multiple applications in a single user environment for '[https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md#using-windows-appvms-in-seamless-mode seamless-mode]' execution of guest programs similar to that of [https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md QubesOS] with hardware GPU acceleration added.

	=== Compatibility With Other ABIs ===		=== Compatibility With Other ABIs ===

Arthur: /* Development Status */

2022-08-22T17:58:46Z

Development Status

← Older revision		Revision as of 17:58, 22 August 2022
Line 31:		Line 31:
	VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in issue [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>724 [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).		VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in issue [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>724 [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).

	While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as ~~driver~~ D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].		While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as disk D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].

Arthur: /* VirtIO-FS */

2022-08-22T17:58:22Z

VirtIO-FS

← Older revision		Revision as of 17:58, 22 August 2022
Line 29:		Line 29:

	=== VirtIO-FS ===		=== VirtIO-FS ===
	VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in issue [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>~~723~~ [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).		VirtIO-FS provides shared filesystem functionality between host and guest using [https://github.com/libfuse/libfuse FUSE] which is ideal for LIME, however in the context of Windows guests the C version of virtiofsd appears unstable (seen in issue [https://github.com/virtio-win/kvm-guest-drivers-windows virtio-win] issue [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/550 <nowiki>550 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/723 <nowiki>723 [closed]</nowiki>], [https://github.com/virtio-win/kvm-guest-drivers-windows/issues/724 <nowiki>724 [open]</nowiki>]). Testing has been performed by members of the OpenMdev community which have shown initial signs of increased filesystem stability under continuous read/write loads using the Rust version of virtiofsd ([https://gitlab.com/virtio-fs/virtiofsd virtiofsd-rs]).

	While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as driver D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].		While many programs can be stored in the VirtIO-FS shared directory some programs such as [https://store.steampowered.com/ Steam] don't want to install programs into the VirtIO-FS drive (seen as a secondary drive in Windows - in LibVF.IO often as driver D:/ if no additional media is attached). This game installation behaviour (which drive the user may place their programs in) may not be in common with other game stores such as the [https://store.epicgames.com/en-US/ Epic Games Store], or [https://www.gog.com/good-old-games Good Old Games (GOG)].

Arthur: /* Win64 Subsystem */

2022-08-22T17:56:25Z

Win64 Subsystem

← Older revision		Revision as of 17:56, 22 August 2022
Line 17:		Line 17:
	These two approaches might be considered two discretely separable models for software development ''<nowiki/>'enterprise'<nowiki/>'' /''<nowiki/>'consumer''' - LIME might then be considered a third approach which preserves the functions of both prior approaches (continual innovation afforded in the kernel-space and backwards compatibility with existing software libraries).		These two approaches might be considered two discretely separable models for software development ''<nowiki/>'enterprise'<nowiki/>'' /''<nowiki/>'consumer''' - LIME might then be considered a third approach which preserves the functions of both prior approaches (continual innovation afforded in the kernel-space and backwards compatibility with existing software libraries).

	ie: A method by which programs "import" a particular explicitly ~~versions~~ ABI similar to how [https://docs.docker.com/engine/api/latest/ Docker] provisions [https://wiki.archlinux.org/title/Chroot chroot] resources like configuration files and ~~supported~~ libraries, but extended to drivers, and the kernel.		ie: A method by which programs "import" a particular explicitly versioned ABI similar to how [https://docs.docker.com/engine/api/latest/ Docker] provisions [https://wiki.archlinux.org/title/Chroot chroot] resources like configuration files and support libraries, but extended to drivers, and the kernel.

	=== Win64 Orchestration ===		=== Win64 Orchestration ===

Arthur: /* Win64 Subsystem */

2022-08-22T17:55:42Z

Win64 Subsystem

← Older revision		Revision as of 17:55, 22 August 2022
Line 13:		Line 13:
	LIME's most immediate use case is around ABI compatibility with Win64 programs in environments without native support.		LIME's most immediate use case is around ABI compatibility with Win64 programs in environments without native support.

	'''A blog post detailing the problems which lead to this work can be read [https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ here]'''. A summary of this post is traditional models for backwards compatibility with Application Binary Interfaces (ABIs) lock-in existing models for kernel/userland development and stagnate architectural innovation. On the other consumer oriented systems like those based on [https://github.com/apple/darwin-xnu Darwin/XNU] break backwards compatibility with user programs version to version of the system in order to introduce architectural improvements in the kernel / userland (for instance micro-service device drivers which may recover from faults at OS runtime).		'''A blog post detailing the problems which lead to this work can be read [https://arccompute.com/blog/why-computers-suck-and-how-openbsd-makes-them-marginally-better/ here]'''. A summary of this post is traditional models for backwards compatibility with Application Binary Interfaces (ABIs) lock-in existing models for kernel/userland development and stagnate architectural innovation. On the other hand consumer oriented systems like those based on [https://github.com/apple/darwin-xnu Darwin/XNU] break backwards compatibility with user programs version to version of the system in order to introduce architectural improvements in the kernel / userland (for instance micro-service device drivers which may recover from faults at OS runtime).

	These two approaches might be considered two discretely separable models for software development ''<nowiki/>'enterprise'<nowiki/>'' /''<nowiki/>'consumer''' - LIME might then be considered a third approach which preserves the functions of both prior approaches (continual innovation afforded in the kernel-space and backwards compatibility with existing software libraries).		These two approaches might be considered two discretely separable models for software development ''<nowiki/>'enterprise'<nowiki/>'' /''<nowiki/>'consumer''' - LIME might then be considered a third approach which preserves the functions of both prior approaches (continual innovation afforded in the kernel-space and backwards compatibility with existing software libraries).