picnoir/nix-gl-host
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
nix-gl-host/src/nixglhost_wrapper_test.py

85 lines
2.7 KiB
Python
Raw Normal View History

Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
import unittest
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
import os
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
from nixglhost_wrapper import CacheDirContent, LibraryPath, ResolvedLib
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
class TestCacheSerializer(unittest.TestCase):
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
def test_hostdso_json_golden_test(self):
lp = LibraryPath(
glx=[
ResolvedLib(
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"dummyglx.so",
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
"/lib",
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"/lib/dummyglx.so",
"031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9406",
)
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
],
cuda=[
ResolvedLib(
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"dummycuda.so",
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
"/lib",
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"/lib/dummycuda.so",
"031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9407",
)
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
],
generic=[
ResolvedLib(
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"dummygeneric.so",
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
"/lib",
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
"/lib/dummygeneric.so",
"031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9408",
)
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
],
egl=[
ResolvedLib(
"dummyegl.so",
"/lib",
"/lib/dummyegl.so",
"031edd7d41651593c5fe5c006fa5752b37fddff7bc4e843aa6af0c950f4b9409",
)
],
path="/path/to/lib/dir",
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
)
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
cdc = CacheDirContent([lp])
json = cdc.to_json()
self.assertIsNotNone(json)
golden_cdc = CacheDirContent.from_json(json)
self.assertEqual(cdc, golden_cdc)
self.assertEqual(cdc.to_json(), golden_cdc.to_json())
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
def test_eq_commut_jsons(self):
"""Checks that object equality is not sensible to JSON keys commutations"""
Re-thinking the host library auto detection mechanism. We made the incorrect assumption that the first DSO we'd stumble upon in the load path would be the most appropriate one for the host system. IE. it'd be the x86_64-gnu-linux on such a system. It turned out not being the case, meaning we can't take such a shortcut and have to handle the case where we end up with multiple same libraries for different archs. This forced us to do two things: 1. It forced us to rethink the cache directory structure. We now have to cache each library directory separately. Meaning we now have to inject the GLX/EGL/Cuda directories of *all* these library directories subpaths to the LD_LIBRARY_PATH. The dynamic linker will then try to open these DSOs (and fail) until it stumble upon the appropriate one. 2. We had to give up on the way we injected EGL libraries using asolute paths. We don't know which DSO is adapted to the wrapped program arch. Instead of injecting the absolute paths through the JSON configuration files, we just stipulate the libraries names in them. We then inject the various EGL DSOs we find through the LD_LIBRARY_PATH, similarly to what we already do for GLX and Cuda.
2022-12-06 14:55:58 +01:00
cwd = os.path.dirname(os.path.realpath(__file__))
with open(
os.path.join(cwd, "..", "tests", "fixtures", "json_permut", "1.json"),
"r",
encoding="utf8",
) as f:
cdc_json = f.read()
with open(
os.path.join(cwd, "..", "tests", "fixtures", "json_permut", "2.json"),
"r",
encoding="utf8",
) as f:
commut_cdc_json = f.read()
with open(
os.path.join(
cwd, "..", "tests", "fixtures", "json_permut", "not-equal.json"
),
"r",
encoding="utf8",
) as f:
wrong_cdc_json = f.read()
cdc = CacheDirContent.from_json(cdc_json)
commut_cdc = CacheDirContent.from_json(commut_cdc_json)
wrong_cdc = CacheDirContent.from_json(wrong_cdc_json)
self.assertEqual(cdc, commut_cdc)
self.assertNotEqual(cdc, wrong_cdc)
self.assertNotEqual(commut_cdc, wrong_cdc)
Introduce a cache system Copying & patching all the DSOs is a time consuming process (~10s on a slow hard drive computer). We definitely don't want to go through it for each process start, we need to introduce a cache. For this cache, we go the concervative way. We're going to "resolve" a DSO name (ie. find the DSO absolute path) and sha256-hash each DSO. We're then going to compare the fingerprints to determine whether or not we need to nuke and rebuild the DSO cache. The cache state is persisted through a JSON file saved in the cache dir.
2022-12-04 10:05:31 +01:00
if __name__ == "__main__":
unittest.main()
Reference in a new issue Copy permalink