2022-03-04 19:28:08 +01:00
{
"cells": [
{
"cell_type": "markdown",
"id": "87adae48-90b8-4132-bccf-fb7ec200df97",
"metadata": {},
"source": [
"# Abstract\n",
"\n",
"We evaluate the performance of a NAR-based, a file-based (uncompressed and xz-compressed) and a casync-based substitution mechanism through 3 scenarios:\n",
"\n",
"1. A
"1. A single derivation version bump (Firefox) impact on the said derivation.\n",
"1. A stable -> unstable channel jump impact on a NixOS machine closure.\n",
"\n",
"For each of these scenarios, we compare how much data the substitution technique required to transfer versus how much data the NAR substitution required to transfer.\n",
"\n",
"Unsurprisingly, the mass rebuild scenario is the one for which we see the biggest improvement: nix-casync cuts down by 48.7% the amount of downloaded data, the xz-compressed file-based substitution cuts down the same amount by 38.4%.\n",
"\n",
"Surprisingly, we see an improvement in the case of a Nixpkgs stable (21.11) -> Nixpkgs unstable (20.05 pre-release) jump: xz-compressed file based substitution cuts down by 18% the amount of downloaded data, Casync by 17.2%.\n",
"\n",
"We see almost no improvements for the derivation bump scenario: xz-compressed file based substitution 1%, Casync 1%.\n",
"\n",
"For file substitution, we can see that the compression is crutial for the overall performance. Uncompressed file-based substitution is consistently 2 order of magnitude worse than the NAR-based substitution.\n",
"\n",
"We can conclude that reducing the substitution granularity, either via Casync of xz-compressed file-based substitution consistently reduces (1% -> 48.7%) the amount of transferred data in 3 different common scenarios."
]
},
{
"cell_type": "markdown",
"id": "f1eb8600-b572-498a-8f8b-15ad6b91d2b4",
"metadata": {
"tags": []
},
"source": [
"# Import Benchmark Data"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f9d5564c-dab3-458d-923f-848e82459c6e",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"def toMb(b):\n",
" return b * (9.537e-7)"
]
},
{
"cell_type": "markdown",
"id": "4b2cb125-b30b-4049-a6d8-ea2468963085",
"metadata": {},
"source": [
"First, let's import the data generated by the `../companeSubsEfficiency` benchmark."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f9e97b01-f776-4439-af58-ac7f1313d1a0",
"metadata": {},
"outputs": [],
"source": [
"results_dir='bench-results'\n",
"def importBenchmarkCSVs(contentDir):\n",
" return {\n",
" \"casync\": pd.read_csv(f\"{contentDir}/casync.csv\",\";\"),\n",
" \"file\": pd.read_csv(f\"{contentDir}/file.csv\",\";\"),\n",
" \"compressed-file\": pd.read_csv(f\"{contentDir}/file-xz-compressed.csv\",\";\"),\n",
" \"nar\": pd.read_csv(f\"{contentDir}/nar.csv\",\";\"),\n",
" }\n",
"\n",
"b = {\n",
" \"massRebuild\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/before-mass-rebuild\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/after-mass-rebuild\"),\n",
" },\n",
" \"channelJump\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/nixpkgs-stable-channel\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/nixpkgs-unstable-channel\")\n",
" },\n",
" \"firefoxBump\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/before-firefox-bump\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/after-firefox-bump\")\n",
2022-03-08 15:26:17 +01:00
" },\n",
" \"gimpBump\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/before-gimp-bump\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/after-gimp-bump\")\n",
" },\n",
" \"emacsBump\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/before-emacs-bump\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/after-emacs-bump\")\n",
" },\n",
" \"openmpiBump\": {\n",
" \"before\": importBenchmarkCSVs(f\"{results_dir}/before-openmpi-bump\"),\n",
" \"after\": importBenchmarkCSVs(f\"{results_dir}/after-openmpi-bump\")\n",
" },\n",
2022-03-04 19:28:08 +01:00
"}"
]
},
{
"cell_type": "markdown",
"id": "81b08f0a-40e6-4379-89a8-8e14102d2c39",
"metadata": {
"tags": []
},
"source": [
"# Methodology\n",
"\n",
"For each of these benchmarks, we're going to evaluate different store path substitution techniques and compare their efficiencies.\n",
"\n",
"The following benchmarks will consist in building a NixOS machine configuration against 2 Nixpkgs commits. We'll simulate a NixOS machine update from the first commit to the second one.\n",
"\n",
"We're going to evaluate the 3 following substitution techniques:\n",
"\n",
"1. **Nar substitution**: This is the substitution model currently used by both the NixOS and Guix project. It consists in `.tar.xz`-ing a full store path. In this benchmark, we'll identify each NAR by its filename, which is derived by the `sha256` sum of their content.\n",
"1. **Casync substitution**: This is an experimental substitution method implemented via the [nix-casync](https://github.com/flokli/nix-casync) project. Here, starting from a NAR, we uncompress it and chunk it in smaller bits. In this benchmark, we'll identify each casync chunk by its filename, which is already derived the `sha256` sum of its content.\n",
"1. **File-based substitution**: This is a substitution method [the Guix](https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00079.html) project brainstormed around. Basically, each store file would be served separately. In this benchmark, we'll identify these files using the `sha256` sum of their content.\n",
"1. **XZ-Compressed File-based substitution**: Similar to the File-based substitution but with each file individually compressed using the xz compression algorithm using the profile 6 extreme.\n",
"\n",
"\n",
"Note: we're using NixOS/Nixpkgs for all these benchmarks. However, since Guix currently use the same substitution mechanism, you can safely assume the same conclusions holds true for it as well."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "2af8dcec-8665-490a-9146-86b81dd108b0",
"metadata": {},
"outputs": [],
"source": [
"def analyse_benchmark_results(i):\n",
" \"\"\"\n",
" Analyse a benchmark results.\n",
" \n",
" :param i: benchmark dataframes. Expecting a \"before\" and a \"after\" dataframe.\n",
" \n",
" Each benchmark simulates the substitutions triggered by transition between two\n",
" nix closures, a \"before\" and a \"after\" one.\n",
" \n",
" For each substitution mechanism, we then simulate what we can re-use and have \n",
" to download by diff-ing the substitution atoms (file, chunk or NAR).\n",
" \"\"\"\n",
" \n",
" _a_nar = i[\"after\"][\"nar\"]\n",
" _b_nar = i[\"before\"][\"nar\"]\n",
" _a_casync = i[\"after\"][\"casync\"]\n",
" _b_casync = i[\"before\"][\"casync\"]\n",
" _a_file = i[\"after\"][\"file\"]\n",
" _b_file = i[\"before\"][\"file\"]\n",
" _a_compressed_file = i[\"after\"][\"compressed-file\"]\n",
" _b_compressed_file = i[\"before\"][\"compressed-file\"]\n",
"\n",
" nar_closure_size = _a_nar[\"Nar Size\"].sum()\n",
" casync_closure_size = _a_casync[\"Chunk Size\"].sum()\n",
" file_closure_size = _a_file[\"Size\"].sum()\n",
" compressed_file_closure_size = _a_compressed_file[\"Size\"].sum()\n",
" \n",
" _nar_merged = _a_nar.merge(_b_nar, how = \"left\", on=\"Nar Name\", indicator=True, suffixes=(\"_after\",\"_before\"))\n",
" nar_dl_size = _nar_merged.loc[_nar_merged[\"_merge\"] == \"left_only\"][\"Nar Size_after\"].sum()\n",
" nar_reused_size = _nar_merged.loc[_nar_merged[\"_merge\"] == \"both\"][\"Nar Size_after\"].sum()\n",
" nar_nar_savings = 0\n",
" \n",
" _casync_merged = _a_casync.merge(_b_casync, how=\"left\", on=\"Chunk Name\", indicator=True, suffixes=(\"_after\",\"_before\"))\n",
" casync_dl_size = _casync_merged.loc[_casync_merged[\"_merge\"]==\"left_only\"][\"Chunk Size_after\"].sum()\n",
" casync_reused_size = _casync_merged.loc[_casync_merged[\"_merge\"]==\"both\"][\"Chunk Size_after\"].sum()\n",
" casync_nar_savings = (nar_dl_size - casync_dl_size) / nar_dl_size\n",
" \n",
" _file_merged = _a_file.merge(_b_file, how=\"left\", on=\"Sha256\", indicator=True, suffixes=(\"_after\",\"_before\"))\n",
" file_dl_size = _file_merged.loc[_file_merged[\"_merge\"]==\"left_only\"][\"Size_after\"].sum()\n",
" file_reused_size = _file_merged.loc[_file_merged[\"_merge\"]==\"both\"][\"Size_after\"].sum()\n",
" file_nar_savings = (nar_dl_size - file_dl_size) / nar_dl_size\n",
"\n",
" _compressed_file_merged = _a_compressed_file.merge(_b_compressed_file, how=\"left\", on=\"Sha256\", indicator=True, suffixes=(\"_after\",\"_before\"))\n",
" compressed_file_dl_size = _compressed_file_merged.loc[_compressed_file_merged[\"_merge\"]==\"left_only\"][\"Size_after\"].sum()\n",
" compressed_file_reused_size = _compressed_file_merged.loc[_compressed_file_merged[\"_merge\"]==\"both\"][\"Size_after\"].sum()\n",
" compressed_file_nar_savings = (nar_dl_size - compressed_file_dl_size) / nar_dl_size\n",
" \n",
" return pd.DataFrame( data = {\n",
" \"Name\": [\"NAR\", \"Casync\", \"File\", \"Compressed File\"],\n",
" \"Closure Size (MB)\": [toMb(nar_closure_size), toMb(casync_closure_size), toMb(file_closure_size), toMb(compressed_file_closure_size)],\n",
" \"Downloaded Size (MB)\": [toMb(nar_dl_size), toMb(casync_dl_size), toMb(file_dl_size), toMb(compressed_file_dl_size)],\n",
" \"Re-used Size (MB)\": [toMb(nar_reused_size), toMb(casync_reused_size), toMb(file_reused_size), toMb(compressed_file_reused_size)],\n",
" \"DL Savings Compared to NAR (%)\": [nar_nar_savings * 100, casync_nar_savings * 100, file_nar_savings * 100, compressed_file_nar_savings * 100]\n",
" })\n",
"\n",
"def gen_perf_pie(dataframe, key):\n",
" idx=mass_rebuild_results.query(f'Name == \"{key}\"').index[0]\n",
" pd.DataFrame(data={\"data\":[dataframe[\"Downloaded Size (MB)\"][idx],dataframe[\"Re-used Size (MB)\"][idx]]},\\\n",
" index=[\"Downloaded\",\"Re-Used\"])\\\n",
" .plot.pie(figsize=(6,6), y=\"data\", ylabel=\"\", title=f\"{key} Downloaded/Re-Used Data\") "
]
},
{
"cell_type": "markdown",
"id": "6e632d28-b1b7-450b-abe0-274337a6dfbb",
"metadata": {
"tags": []
},
"source": [
"# Benchmark Scenarios\n",
"\n",
"\n",
"## 1. Mass Rebuild\n",
"\n",
"Let's build the same NixOS machine description using two Nixpkgs commits: one before and one after the [staging next 2021-12-03](https://github.com/NixOS/nixpkgs/pull/148396) iteration merge to master. This staging iteration contains, among other things, a `curl` version bump. That `curl` version bump triggers a almost entire Nixpkgs mass rebuild: both `nix` and `stdenv` are depending on it.\n",
"\n",
"This mass-rebuild scenario represents a long-standing issue in terms of substitution performance."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "85c720c5-4688-4d7e-84d3-4006700a69fb",
"metadata": {},
"outputs": [],
"source": [
"mass_rebuild_results = analyse_benchmark_results(b[\"massRebuild\"])"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "401c1c9d-b264-4266-ac92-de660f4577bc",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
" <td>386.060194</td>\n",
" <td>372.989991</td>\n",
" <td>13.070203</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
" <td>608.652010</td>\n",
" <td>192.194724</td>\n",
" <td>416.457286</td>\n",
" <td>48.471882</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
" <td>1652.194177</td>\n",
" <td>705.665216</td>\n",
" <td>973.347687</td>\n",
" <td>-89.191462</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
" <td>476.522900</td>\n",
" <td>229.488728</td>\n",
" <td>247.034172</td>\n",
" <td>38.473221</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
"0 NAR 386.060194 372.989991 \n",
"1 Casync 608.652010 192.194724 \n",
"2 File 1652.194177 705.665216 \n",
"3 Compressed File 476.522900 229.488728 \n",
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
"0 13.070203 0.000000 \n",
"1 416.457286 48.471882 \n",
"2 973.347687 -89.191462 \n",
"3 247.034172 38.473221 "
]
},
2022-03-08 15:26:17 +01:00
"execution_count": 5,
2022-03-04 19:28:08 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mass_rebuild_results"
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 6,
2022-03-04 19:28:08 +01:00
"id": "8a800b9f-fb85-4b4f-a713-588971ecdc9b",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAGGCAYAAABbgxaRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1uElEQVR4nO3de5hVdd3//+dbUMFDiQp+UQxQ8SyiIXbnITykpgZ2f9M8pmWa3mbZWStL799N2f01s7PSSU2TuCvTrDyEt1paKaaheCQ1RFCQFM8E+P79sdbgZpgZBtfs2TOzn4/rmmvv9VmH/d5r75l57c/6rLUjM5EkSZL0xqzR6AIkSZKk3sxALUmSJFVgoJYkSZIqMFBLkiRJFRioJUmSpAoM1JIkSVIFBmqpEyJifETMaXQdfUVEnBARf6zTtjMitmpn3iYRcWtEvBARX6vT418SEf9Vj233RlV/dyLixYjYorzf4b7t6LXvat39NyEiBkfEQxExoJy+OSI+1E2PfUxE3PAG1uu2GjsrIu6IiB0aXYf6HgO1mkZEXB8R/9lG+8SIeCoi+jeirtXRBeHkhIhYVoaUFyPisYj4cURs3ZV19mAnA88Ab8rMT1bdWD0/GJTbv7kMiTu3av9V2T6+Xo/d6vFq3zfPR8TfIuLQ7njszFwvMx+tup2IeDwi9m/VVtfXr4sf50zgx5n5alfUtDoy84rMPKA7H7OtMN5FH5jOB1b6PyBVZaBWM7kEOC4iolX7ccAVmbm0+0tqiD9l5nrAm4H9gVeAuyJix8aW1S2GA/fnG/hGqwZ+4HoYeH9NHRsBbwMWdHMdLe+bDYDvAlMiYoNurqEpRcTawPHA5Y2upbeq+f29BtgnIoY2sh71PQZqNZNfARsCe7U0RMQg4FDgsohYOyIujIi55c+F5T+ylbTuKak9FN3SixwRn4mI+RExLyIOi4iDI+LhiPhnRHyuZt01IuLMiPh7RCyMiKkRsWEbj7ku8Dtg05oe5k1Xp+5ambksM/+emf8B3AKcU/NYEyJiZkQ8V/YUbVe2fyAifl2z3KyImFoz/UREjKnZR6dExCMR8WxEfKeNDzMt6709Iu6MiEXl7dtr5n0gIh6IYpjGoxHx4Vbrfrrcx3Mj4oPtPd+IuIQilHym3Hf7d7Tval7Hz0bEU8CPW21vO+Ai4N/K7T1XM3tQRPymrPkvEbFlzXrbRsSN5fvgoYg4or2aS1cA74uIfuX0UcBVwL9qtjkuIv5Uvl7zIuLbEbFWOS8i4uvle3FRRMxo+fBUvifvL+t8MiI+tYpayMzXgJ8A6wKjyu2sHRHnR8TsiHg6Ii6KiIGt9tfnIuKZKHqKj6lpX6EnMlr15rb+XWu1zU699p0RESPKxzq53N68iPhkzfyBUfyePxsR9wO7tVq/5Xf4hXKfvqdsb/N90pl9VmN34LnMbPfoVER8sPw9eTaKo3HDy/bKr3/ta9LR9tqxZRTDLBZFxNVR87ctIt4WEbeX79u/RXnEJSImUfyd/na5z74dEbeWq/2tbHtfueyhEXFPuY3bI2J0zfYfj+L3dwbwUkT0L3v47wK6tcddTSAz/fGnaX6A7wM/qJn+MHBPef8/gT8DQ4DBwO3A/1fOGw/MqVkvga1qpi8B/qtm2aXAF4E1gZMoehN/CqwP7AC8CmxRLn9G+bjDgLWBi4Er26l/hTpWVXcb658A/LGN9g8CT5f3twZeAt5Z1v8ZYBawFrAF8BzFh/GhwD+AJ8v1tgCeBdao2UfXUvRovqXcBwe1roPiQ86zFEcK+lMExmeBjcr5hwBbAgG8A3gZ2LWcdxDwNLAjRcD7aevXptXzXP46dfI1Xwp8tXxdBnZmf5aP8U9gXPl8rgCmlPPWBZ4APlDO25ViCMoO7dR7M/Ah4AbgXWXbHcC/AXOA8WXbWyl6rfsDI4AHgDPKeQdSBIgNyn24HTC0nDcP2Ku8P6hlv3b0PIF+wGkUgX5I2XYhRc/fhhTv8V8DX2m1Hy8o9+M7KN5f29Q+x/b2ae3ryYq/Z6v72j8O7N/B8xpRrn9lub2dKN6z+5fzzwP+UD7HzYH7WPFvwuHAphS/G+8rn+PQtp7TqvZZG7WfBvymrfdGef8wit/R7cr3wBeA2+v0+re7vXbev0/WvEa/AC4v520GLAQOLvfZO8vpwW29L9r5u7srMJ/iA0c/ig/MjwNr17zm95Sv18Ca9b4JXNBWzf7480Z/7KFWs7kUOLymJ+j9ZRvAMcB/Zub8zFwAnEsR8t6IJcCkzFwCTAE2Br6RmS9k5kxgJtDSk/Jh4POZOSczF1P0FL83Oj/EoCvqnkvxjx2KMPCbzLyxrP98YCDw9izGsr4AjKEIRtcDT0bEtuX0H7LowWxxXmY+l5mzgf8t12vtEOCRzPxJZi7NzCuBB4F3A2Tmb7LoSc/MvIUiXLYcZTiCYlzpfZn5EjW97J20qn33GvClzFycma+sxnZ/mZl3ZDGM6Apef96HAo9n5o/L5/pXipDx3lVs7zLg/RGxDbBBZv6pdmZm3pWZfy63+TjFh7J3lLOXUAS2bYHIzAcyc17NvO0j4k2Z+WxZT3veVvauvkrxnjg2M+dHRFB8aPx4Zv4zM18Avgwc2Wr9s8v9eAvwG4rXroqqr317zs3MlzLzXoqjEkfVPN6k8jk+QRHKlsvM/8nMuZn5Wmb+DHiE4kPVSlZjn7XYgOL3rj0fpgjjD5TvuS8DY8pe6q56/Vt0tL22/KTmNTobOCKKoy3HAr/NzN+W++xGYDpFwO6sk4CLM/MvWRxxuxRYTPHhssU3M/OJVr+/L1DsU6nLGKjVVDLzjxS9ThOjuHLAbhQ9W1D0Lv2jZvF/lG1vxMLMXFbeb/lD/nTN/FeA9cr7w4GrykOWz1H0Li4DNunkY3VF3ZtR9KqutL0yID9RLgPF8JDxwN7l/Zspwts7yulaT9Xcf5nXn3NH9VNObwYQEe+KiD9HMUTiOYp/uBvXrPtEq/VWx6r23YJ8YyeBtfe8hwO7t7zW5fM5Bvg/q9jeL4F9gdMphlusICK2johrozi59nmKQLUxQGbeBHwb+A7wdERMjog3lav+X4r9+Y+IuCUi/q2DGv6cmRtQ9GRew+sfagYD61CMw295TteV7S2eLQNViyq/Wy1W97VfSnHEpdaaFAGxVuttttTZ4eNFxPtrhh48R9EruzFt68w+q/UsRYhtz3DgGzXb+idF7/FmXfj6A6t8P7Wl9T5bk2K/DKfo3Kj9XdiT4shXZw0HPtlqG5uz4nvriTbWW5/iSJvUZQzUakaXUfRMHwfckJktQXcuxR/oFm8p29ryMsU/xBarCkQdeYLicP4GNT8DMvPJNpZt62S61am7Pe+hOJy90vbK3rTNKQ7dwuuBeq/y/i20H6g7o3X9UDyHJ6MYz/wLih7RTcpA91uKsADFIevNW61X5bFb77tVnby4uic3PgHc0uq1Xi8zT+3wQTJfphg/fyptBGrgexS9+qMy803A53h9H5GZ38zMt1IMN9oa+HTZfmdmTqQY8vIrYCqrkJkvAv9BcYLvLhRDVl6hGLbS8pzenMUJjC0GRXEOQIva/fwSb+x3aXVf+9kUwzpqjWTlIN56my11tvt4ZU/w94GPUAxV2oBiSEjLa9D6fdKZfVZrBsXr1p4ngA+3el8NzMzboWtf/462147W+2wJxfN/gqL3urbmdTPzvJaH6UQpT1AcNajdxjpZHOVaXm4b620H/K0T25c6zUCtZnQZxdUtTuL14R5QjJ38QhTXe92YYgx0e2fV3wMcHRH9IuIgXj+8/kZcBEyqOYlocERMbGfZp4GNIuLNb7Du5craR0bEtygC8rnlrKnAIRGxX0SsCXyS4jDq7eX8W4B9KMYkzqEI4gcBGwF3d/ZJ1/gtsHVEHB0R/cuTjbanGH+9FsW42wXA0oh4FyueTDQVOCEito+IdYAvreZjv6F9V+NpYFiUJwB2wrUUz/W4iFiz/NktypM+V+FzwDvKIR2trQ88D7xYDr9ZHtDL7e9evpYvUQzZWBYRa0VxfeE3ZzG053mKIyOrlJkLgR8AXyyPYHwf+HpEDCkfc7OIOLD
"text/plain": [
"<Figure size 864x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_ = mass_rebuild_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"Downloaded Size (MB)\",title=\"Volume to Download for the Mass Rebuild Update (less is better)\", xlabel=\"\", ylabel=\"Size in MB\")"
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 7,
2022-03-04 19:28:08 +01:00
"id": "326255a1-a681-4405-9a40-889651392948",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtYAAAGGCAYAAABfdsasAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0/klEQVR4nO3debxd873/8dcniUrMLTEPQbWEhJJQQxVpS823JGKMn5ZbpTWVGkrRal3TpZfellKtCqKCFK0aamyJcIPEUClKSCWCiCBCPr8/1jrHTpxzcsI+Z53h9Xw8ziN7r/Gzh5Pz3t/9WWtFZiJJkiTpk+lRdQGSJElSV2CwliRJkurAYC1JkiTVgcFakiRJqgODtSRJklQHBmtJkiSpDgzWkioVEW9FxFpV19EVRcQ2ETG56jrqLSJ+FhFHVl3H/CLiSxHx9MdY7/KI+Elb1PRxRcToiNih6jqkzsZgLXVREfF8RLwTETMj4o2I+FtEfDsietQs0+o/6BHxzYh4qtzeKxFxc0Qs+UnrzMwlMvPZT7qdhRERn4qIUyPimYiYVT5Xl0VEv/aso2oRkRHx2Y+5br9y/Zvnm/77iDh1vmlrRsTciPhFMzXMKj9gvRQR50VEzxb22xc4APjVx6m7LWXmvZn5+fbcZ1O/w+X7+SufcNNnAmd8wm1I3Y7BWuradsnMJYE1KP5Q/gC4dGE3EhFfBn4K7F1ubz1gVD0LbWd/AHYF9gGWBjYEHgaGVFnUgrQUOCv0xYjYcgHLHAC8DgyPiEWbmL9hZi4BfBnYCziohW0dCNySme98nGIXRkT0aut9dDRR6JGZY4GlImJQ1TVJnYnBWuoGMnNGZo6hCC0jImKDhdzEYODvmfl/5fZey8zfZuZMgIjYKSL+LyLejIgXa0csI+LPEXF47cYi4tGI+EZ5u3HUtBx9u6gcDZ8ZEQ9GxNo1630tIp6OiBkR8YuIuDsivlXO+2x5f0ZEvBoR1zT1QMqRvK8Cu2XmQ5n5fvn8XJSZl5bLrBwRYyLitYiYFBEH16x/akRcW47MzoyIxyPicxFxQkRMLR//12qWv6tsXRhb1nZjRHymZv61EfHvct49EbF+zbzLI+J/I+KWiJgFbFvWdl1ETIuI5yLiezXL9ynXeT0inihftyZFxD3lzUfL0eK9yukHl4/5tfI5WLm5bZTOAhb0rccBwA+BOcAuzS2UmZOA+4GNWtjW14G7ax7HNhExOSKOK5//KRGxe0TsGBH/KB/HiTXLLxoR50fEy+XP+Q1hv2ZbP4iIfwO/iYgeEXF8RPwzIqZHxKja169WzNd6U27npfJ98nREtPTBbbmIuK1c9u6IWKNmO+uW814rtzOsnH4IsC9wXPka/jEirgBWB/5YTjuuXPaLUXxr9UYUv3/b1Gz/rog4IyLuB94GGlqz7gJ2aqFmSfMxWEvdSDkKNRn40kKu+iCwfUScFhFbxkdHHWdRhKdlKP4QHxoRu5fzRgJ7NywYEf0pRtBvpml7A6cBnwYmUX4dHRHLUYw0nwAsCzwNbFGz3o+Bv5TrrQr8TzPb/wowNjNfbOHxXkXxPK0M7An8dL5QtAtwRbmv/wNupfj/dBXgdD7apnAAxSjsysD7wM9r5v0JWAdYHngEuHK+dfeheA6WBP4G/BF4tNzXEODIiNi+XPZHwNrlz/bAiOYeYGZuXd7csGzHuSYitgN+BgwDVgL+BVzd3DZKFwGfi2ZaDyLiSxSvx9UU33Ic0NyGImJdivfmpBb2N4Dita+1ItCb4jk5BbgE2A/YpNzeKfFhH/9JwBcpwvuGwKYUob92W5+heI8eAnwP2J1iNH1lipH3i1qor+GxfB44HBhcfsuzPfB8C6vsS/EeXg4YT/k+iIjFgdsofo+Wp/j9+EVErJ+ZF5fLnVW+hrtk5v7ACxTfVi2RmWdFxCoUv28/KR/b94HromirabB/+XiXpHjdAZ4snyNJrWSwlrqflyn+uLZaZt4LfAPYmOIP9PSo6YXNzLsy8/HMnJuZj1EE0y+Xq18PbFQzArcvMDozZzezu9GZOTYz36cIDRuV03cEJmbm6HLez4F/16w3hyIMrZyZ72bmfc1sf1lgSnOPNSJWA7YCflBuZzzwa4rg0eDezLy1rONaoC9wZmbOoQiQ/SJimZrlr8jMCZk5CzgZGFbz3F2WmTPL5+NUYMOIWLpm3Rsz8/7MnEsRKvtm5umZ+V7Zm34JMLxcdhhwRvmNwovMG+BbY1/gssx8pKznBGDzaLn3/F2K4N/cqPUI4E+Z+TpFOPx6RCw/3zKPlCPyT1KMkn6kF7vGMsDM+abNoXjcDc//csAF5fM6EZgIDKx5jKdn5tTMnEbxIa72tZ0L/CgzZ5ftJv8JnJSZk2teoz1jwW0iHwCLAv0jYpHMfD4z/9nC8jdn5j3lPk6ieN5XA3YGns/M35TfrjwCXEfxga+19qNon7ml/B29DRhH8TvV4PLMnFjuY045bSbF8y2plQzWUvezCvDawq6UmX/KzF0oQvluFL2uDW0Ym0XEX8v2hBnAtynCDWW7yM18GP6G89FR2Vq1YfltYIny9spA4yhzZibFqHKD44AAxkbExIhork93OsVobHNWBl5raHMp/YvieWvwSs3td4BXM/ODmvvU1E1t3eW2FqH46r9nRJxZthm8yYcjmss1s+4awMrl1/lvRMQbwInACjW1z7+vhbFy7TqZ+RbF87VKs2sULgFWiIh52jwiog8wlPL1zsy/U4ym7jPf+htTPF97AZsBi7ewr9cpRlVrTW/i+Z//Nap9H9U+L/8qpzWYlpnv1txfA7i+5vl+kiI0r0ALyraWIymC+NSIuHoBbTW17+23KH5HVy73v9l8r/m+FCPrrbUGMHS+bWzFvL8HTX2DsyTwxkLsR+r2DNZSNxIRgylCUnOjuQtUjnjdAdwJNPRqjwTGAKtl5tLALylCboOrgL0jYnOgD/DXj7HrKRQtBUBxkFXt/cz8d2YenJkrU4wy/iKaPuPF7cCmEbFqE/OgHNGPec94sjrw0seoucFq821rDvAqRcDcjaI9ZWmgX7lM7XOXNbdfBJ7LzGVqfpbMzIaRxylN7GthvEwRwooiijaEZVnAYy9HOE+jaGWorf0/gKUoXot/l33Lq9BEO0gWRgF/p2jnaM5jwOda9WiaNs9jpHiOXq4tZb7lXwS+Pt9z3jszF/h+yMyRmblVub8E/quFxRtft4hYguID7Mvl/u+eb/9LZOahzdTb3GO4Yr5tLJ6ZZ7awDhQHKT/a4oOUNA+DtdQNRMRSEbEzxdfkv8/Mx2tm94yI3jU/n2pi/d0iYnhEfDoKm1K0ejxQLrIkxSjvu+W8+Uckb6EIF6cD15RtDQvrZmBAFAem9QIOo2bULiKG1oTl1ymCwgfzbyQzb6foWb0+IjaJiF4RsWQUpyI8qGyh+Bvws/L5GAh8k5ZH2Rdkv4joHxGLUTwHfyhHWJcEZlOMCi9GceaVlowF3iwPiutTjnhvUH5ggqKH+YTydVoV+O4CtvcKHx6oBsUHpP8XERuVffQ/BR7MzOdb8RivoGh9qD338QjgMooWlo3Kny0pWoMGNLOdM4FDIqK5Edlb+LDN6OO4CvhhRPQt+/ZPAX7fwvK/BM5oaGUq19ttQTuJiM9HxHbl8/guxaj5R96PNXaMiK3K378fUzzvLwI3UfSw7x8Ri5Q/gyNivXK9+V/Dpqb9HtglIrYv3zO9ozjQsrkPlw2+THEMgKRWMlhLXdsfI2ImxYjVScB5wP+bb5njKf7oN/zc2cR2XgcOBp4B3qT4Q312ZjaEze8Ap5f7OoX5TsVX9o2OphiZHflxHkhmvkrRVnAWRRDtT9En2tCrPRh4MCLeohg9PyIzn2tmc3tSBLRrgBnABGAQxWg2FAeI9aMYMbyeouf2to9Td+kK4HKKNpfeFAfEAfyOohXhJeAJPvyg0qQyjO9CEVCfoxj1/jXFaDcUo8b/Kuf9pdxvS04Fflu2Bwwrv4k4maKHdwrFQZDDW1h//tp+RNm/Xx4wNwQ4v/w2oeHnYeDPNHNgZfmh727g2GZ29TuKENqnNXU14ScU75vHgMc
"text/plain": [
"<Figure size 864x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_ = mass_rebuild_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"DL Savings Compared to NAR (%)\",title=\"DL Savings Compared to NAR (more is better)\", xlabel=\"\", ylabel=\"Savings in %\")"
]
},
{
"cell_type": "markdown",
"id": "49e237d1-6ed7-4e2b-bc9d-f51d9a9eed2f",
"metadata": {},
"source": [
"We can see a massive performance gain for both Casync (48.4%) and xz-compressed files (38.4%). We can also see that compression plays a massive role in terms of substitution performance: the uncompressed files are doing almost 90% worse than the plain NAR substitution."
]
},
{
"cell_type": "markdown",
"id": "2ca085b5-a768-468c-ab8a-76bde3de6818",
"metadata": {},
"source": [
"## 2. Firefox Bump\n",
"\n",
"In this scenario, we're going to simulate a Firefox update. We took the Firefox 97.0 -> 97.0.1 bump [7e23a7fb8268f16e83ef60bbd2708e1d57fd49ef](https://github.com/NixOS/nixpkgs/commit/7e23a7fb8268f16e83ef60bbd2708e1d57fd49ef) as a test example."
]
},
{
"cell_type": "code",
"execution_count": 16,
2022-03-08 15:26:17 +01:00
"id": "318921b6-bd97-46e5-9aa4-b89e01831bb1",
2022-03-04 19:28:08 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
" <td>219.351904</td>\n",
" <td>56.431531</td>\n",
" <td>162.920373</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
" <td>342.924949</td>\n",
" <td>55.870973</td>\n",
" <td>287.053976</td>\n",
" <td>0.993342</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
" <td>946.853440</td>\n",
" <td>249.259020</td>\n",
" <td>723.274748</td>\n",
" <td>-341.701675</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
" <td>260.424006</td>\n",
" <td>55.829510</td>\n",
" <td>204.594495</td>\n",
" <td>1.066817</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
"0 NAR 219.351904 56.431531 \n",
"1 Casync 342.924949 55.870973 \n",
"2 File 946.853440 249.259020 \n",
"3 Compressed File 260.424006 55.829510 \n",
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
"0 162.920373 0.000000 \n",
"1 287.053976 0.993342 \n",
"2 723.274748 -341.701675 \n",
"3 204.594495 1.066817 "
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2022-03-08 15:26:17 +01:00
"firefox_bump_results = analyse_benchmark_results(b[\"firefoxBump\"])\n",
2022-03-04 19:28:08 +01:00
"firefox_bump_results"
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 10,
2022-03-04 19:28:08 +01:00
"id": "1e1eb388-b4ab-400a-90a9-f46bfe4f4541",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAGGCAYAAABbgxaRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAyXUlEQVR4nO3deZwcdZ3/8deHBEiAqEASFggk4T5DwBBcIRgOAQEB96dyGeUQhBUEj/XaReKuCLqAKB4QERHBYFZEEREQkNMDEsFAuAUMASQhQrhjEj6/P6pm6Awzk0lqenpm+vV8PPrRddenq2p63l39rerITCRJkiStmJUaXYAkSZLUlxmoJUmSpAoM1JIkSVIFBmpJkiSpAgO1JEmSVIGBWpIkSarAQC11QURMjIg5ja6jv4iIIyLitjotOyNikw7GrRMRt0TEixFxVp3Wf1FEfKUey65Zx6yImFhh/q9ExLMR8ffuq6pnRMSGEfFSRAxodC09LSJOj4iTy+4efU9akWOuN75vRsQBEXFZo+tQ/2OgVtOIiGsj4r/bGX5gRPw9IgY2oq7lUfUfVBlkl5SB5KWIeCwifhgRm3Vnnb3YscCzwFsy89NVF1bPDwbl8m+KiNdq9tdLEfGvmbl1Zt60gsvcAPg0sFVm/ks31bl+RCyOiI3bGXdFRJzZHesByMzZmblGZi7prmW2KD+MvVxu52cjYmpEvK2717MiImIY8GHg/Easv8oxtyLae6+LiMkRcUmV5WbmlcA2ETGmUoFSGwZqNZOLgEkREW2GTwIuzczFPV9SQ/whM9cA3grsCbwKzIiIbRpbVo8YCdyXK/CLVg38wHVCGSBbHn/obOIu1DkSmJ+Zc7urwMx8EriB4m+ptpa1gH2BHy3P8hr84Xa78u9jI2BNYHIDa6l1BHB1Zr7a6EL6qprjairFh2up2xio1Ux+AawFTGgZEBFrAvsDF0fEqhFxTkQ8VT7OiYhV21tQ22YFtV/zt5xZiYjPRsTciHg6Ig6KiH0j4qGI+EdEfLFm3pUi4vMR8deImB8R08og0nadqwO/AdarOVu53vLUXSszl2TmXzPz34GbqQkO5deisyLi+fIs6Zbl8CMj4lc10z0SEdNq+p+IiLE12+i4iHg4Ip6LiO+082GmZb53RsSdEbGgfH5nzbgjI+L+KJppPBoRH2sz73+U2/ipiDiqo9cbERcBHwE+W267PTvbdjX78XNRNI34YZvlbQmcB/xrubzna0avGRG/Lmv+U+2Z24jYIiJ+Wx4HD0bEBzuquZPX8nhE7Fl2T46In0XEJRHxAnBERLw1In5Qbpcno2jiMaCc57e8cQxdVC6jo/39uYj4Y0sQiYjjy+kGtVPWj2gTqIFDgFmZeU95rF4eEfOi+GbkEzWvp73XMD4ipkfECxHxTEScXU47qjy2WmpaLyKuLLfnIxFxTJvlTouIi8t9MSsixnVlG2fmC8CVwFbtbfea5V/Spq4jy7+D58rjf8eImFlu22/XzHtERNweEeeWx/0DEbFHJyW9h+LvtF3L2L4dbctB5TafX9Z3Z0Ss08Hya4+5dpfXSW1fjOKM/+MRcXjN8FUj4syImF0u57yIGBztv9cdBnwROLjs/0u5jHaP9Tbb+BsR8Q/eeI+7Cdivs5ql5ZaZPnw0zQP4PnBBTf/HgLvL7v8G/ggMB4YBvwf+pxw3EZhTM18Cm9T0XwR8pWbaxcCXgJWBY4B5wE+AIcDWwGvARuX0J5frHQGsSvGV7tQO6l+qjmXV3c78RwC3tTP8KOCZsnsz4GXg3WX9nwUeAVahOGv3PMWH8XWBvwFPlvNtBDwHrFSzja4C3gZsWG6DfdrWQfEh5zmKMDYQOLTsX7scvx+wMRDAu4BXgB3KcfsAzwDbAKuX23ipfdPmdbbupy7u88XA18r9Mrgr27Ncxz+A8eXruRS4rBy3OvAEcGQ5bgeKJihbd1DvTcBH2xn+OLBn2T0ZWAQcVO6XwRQfHs8v1zccuAP4WAfHcmf7eyXglnIdm5b7ZfsOah0MLAB2qRn2B4rjeyVgBsXfRMtx9Ciwdyev4Q/ApHL8GsA7yu5R5T4eWPbfDHwXGASMpTjO9qhZ7msUZ8kHAKcDf+zk/aH12KE4O30d8N/tbfea5V/Spq7zylr2Ktf9i3IfrA/MBd5Vc+wsBj5ZbveDy+23Vge1zQN2bO+9oAvbt6Nt+THgV8Bq5fZ5O0VzqPbW3/raO1peB+9Xi4GzKf6G3kVxrG1ejj+H4kPLWhTvjb8CTu/kva51e9cM+wUdH+st2/hEir+3wTXvOdnRa/XhY0UeDS/Ah4+efAC7lP+0Wt5Ybwc+WXb/Fdi3Ztq9gcfL7qXe3Fl2oH4VGFD2Dymn36lm+hnAQWX3/ZQBoOxflyJcDGyn/vb+yXRYdzvzH0H7gXofYFHZfQowrWbcSsCTwMSy/wmKIHgIMKX8B7YFRUi8ss02qg1X04DPt62DIkjf0aaePwBHdPAafgGcVHZfCJxRM26ztvumzbyt+6mL+/yfwKBOjqc3bc9yHbUf2vYFHii7DwZubTP9+cCpHSz/JooPEM+Xjz+Xwx9n6UB9S8086wALqfkAQPEh5XcdHMvL2t+jKD4g3A98YRl/XxcAU8ruTcvtNxzYCZjdZtovAD9s7zWUw24BvgwMbTN8VLmPBwIbAEuAITXjTwcuqlnu9TXjtgJe7aT+BF4ot/US4AFg/Zrxrdu9ZvltA3Xt9POBg2v6LwdOrjl2ngKiZvwdlEG1ndoWAVvU9Lfuxy5s34625VEUHyLHdLZf2znm2l1eO/NMpAi0q9cMm1Yec0ERrjeuGfevwGPtHadtt3cXj/Uj2m6XcvjK5b7acFmv24ePrj5s8qGmkpm3UZzpOTAiNgJ2pDirCbAexRnXFn8rh62I+fnGRVMtbR6fqRn/KsWZHSjatF5RfuX6PEVwWULxz6IruqPu9SlC05uWl5mvU4To9ctBN1P8s9u17L6J4szTu3jzV9K1d5F4hTdec2f1U/avDxAR7ymbHfyj3D77AkNr5n2izXzLY1nbbl5mvracy4SOX/dIYKeWfV2+nsOBzi4O/ERmvq187NDBNLXbYCRFYHi6Zh3nUwTb9nS6vzPzceB3FIHxO53UCUWzjw+WTUImAddk0VZ7JMXX97Wv+4ssfYw/0WZZR1N8QHqgbIqwfwe1/yMzX6wZ1nrslNrui0HReRvtHTLzbRRnmb8H3NpBE5eOtP077+jvHopvd7JN7R397T5H8eG8Pcvavh1tyx8D1wKXRdHk6esRsXLnL6/T5bVbd2a+XNPf8hqHUZwZn1FT8zXl8K7qyrHe9riCN7bj88uxLqlTBmo1o4sprpafBFyXmS3/8J6ieINusWE5rD2vUPwzaFHlbglPAO+pCU1vy8xBWVzo1Va2M2x56u7I+4Bb21teRATFmcCWeloC9YSy+2Y6DtRd0bZ+KF7Dk1G0Z74cOBNYpww6V1Oc3QJ4uqytdr4q62677drb3izH+LaeAG5us6/XyMzjl3M5ndXxBMVZu6E163hLZm7dwbyd7u+I2JfizOENwP92WkTmrRRnZQ8EPkTxt9ZS02NtXveQzNy3g9dAZj6cmYdShKOvAT8r29a2rX2tiKgNmhvyxrG6wjJzEcUZ99EUTYqgOKPaXX/3AOuX27tFZ3+7MylCbHs63b4dbcvMXJSZX87MrYB3UlxP8uFlFd3FfdNizTbjWl7jsxQfMLauqfmtWVwQCu3/bbUd1pVjvb3lbEnxTdQLnb5QaTkYqNWMLqa4u8UxLH33ganAf0XEsIgYStEesaNbNN0NHBbFhV77UITJFXUecFpEjITi9lgRcWAH0z4DrB0Rb13BuluVtY+OiHMpAvKXy1HTgP0iYo/ybNWnKf5p/b4cfzOwG8XXrHMogvg+wNrAXV190TWuBjaLiMMiYmBEHEzx1fxVFO1BV6X4VmFxRLyHom1qi2kUF7BtFRGrAacu57pXaNvVeAYYERGrdHH6qyhe66SIWLl87BjlRYDdITOfpmj7e1ZEvCWKi143joiOjtEO93e5TX4AfJTigs73lgG7MxdThKy3UbS
"text/plain": [
"<Figure size 864x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_ = firefox_bump_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"Downloaded Size (MB)\",title=\"Volume to Download for the Firefox Version Bump (less is better)\", xlabel=\"\", ylabel=\"Size in MB\")"
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 11,
2022-03-04 19:28:08 +01:00
"id": "d3886df5-9aaf-4146-8889-5c63c825e916",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtwAAAGGCAYAAABIVFZlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3sUlEQVR4nO3debhVZfn/8fcN+BWcy3CCFDVNEZAUZzNNv2nOaSCO+LMiLXMqSzPNHMqvQ6kNpmaZBs44hJqp5ZiGaMjgiFOSlISKOCHK/ftjrXPc4DmHA7jYZ3i/rmtf7P2s6d777MP57Gc/61mRmUiSJEmqRpd6FyBJkiR1ZAZuSZIkqUIGbkmSJKlCBm5JkiSpQgZuSZIkqUIGbkmSJKlCBm5JbVJEvBERa9W7jo4oIraNiCn1ruOjFhE/iYij6l3HvCLisxHx5EJsd2lEnFZFTQsrIkZFxE71rkNqbwzcUicTEc9HxNsRMTMiXouIv0XEoRHRpWadVv+hj4ivRMQT5f7+ExE3R8Syi1pnZi6Tmc8u6n4WRET8T0ScHBFPR8Sb5Wv124joszjrqLeIyIj41EJu26fc/uZ52v8QESfP07ZmRMyJiF81U8Ob5Qevf0XETyOiawvH7QkcBFy4MHVXKTPvzcxPL85jNvU7XL6fd1jEXZ8BnL6I+5A6HQO31DntlpnLAmtQ/AH9HnDJgu4kIj4H/BjYt9zf+sDVH2Whi9m1wO7AfsDywIbAw8D29SxqfloKonW0eURsNZ91DgJeBYZGxJJNLN8wM5cBPgfsAxzSwr4OBm7JzLcXptgFERHdqj5GWxOFLpk5BlguIgbVuyapPTFwS51YZs7IzJsowsywiOi3gLvYBHggM/9R7u+VzPx9Zs4EiIhdIuIfEfF6RLxY28MZEX+KiMNrdxYRj0bEXuX9xl7Wsrful2Xv+cyI+HtErF2z3Rci4smImBERv4qIuyPiq+WyT5WPZ0TEfyPiqqaeSNnz97/AHpn5UGa+V74+v8zMS8p1VouImyLilYiYHBFfq9n+5Ii4puzJnRkREyJi3Yg4PiJeLp//F2rWv6scAjGmrO3GiPh4zfJrIuLf5bJ7ImKDmmWXRsQFEXFLRLwJbFfWdl1ETIuI5yLiiJr1e5TbvBoRj5U/tyZFxD3l3UfL3uV9yvavlc/5lfI1WK25fZTOBOb3LclBwA+A2cBuza2UmZOB+4GBLezri8DdNc9j24iYEhHfLV//qRGxZ0TsHBFPlc/j+zXrLxkR50bES+Xt3IYPATX7+l5E/Bv4XUR0iYjjIuKZiJgeEVfX/vxqxTxDeMr9/Kt8nzwZES19oPtERNxernt3RKxRs5/1ymWvlPsZUrYPB/YHvlv+DP8YEZcDqwN/LNu+W667eRTfcr0Wxe/ftjX7vysiTo+I+4G3gIYhXncBu7RQs6R5GLglUfZaTQE+u4Cb/h3YMSJ+FBFbxYd7Kd+kCFUrUPyBPiwi9iyXjQT2bVgxIvpS9LjfTNP2BX4EfAyYTPm1dkR8gqJn+nhgReBJYMua7U4F/lxu1xv4eTP73wEYk5kvtvB8r6B4nVYDvgz8eJ6wtBtweXmsfwC3Ufw/2ws4hQ8PdziIotd2NeA94PyaZbcC6wArAY8AI+bZdj+K12BZ4G/AH4FHy2NtDxwVETuW6/4QWLu87QgMa+4JZuY25d0Ny2E9V0XE54GfAEOAVYEXgCub20fpl8C60cwQhoj4LMXP40qKb0UOam5HEbEexXtzcgvH60/xs6+1CtCd4jU5CbgYOADYuNzfSfHBeQInAJtThPoNgU0pPgzU7uvjFO/R4cARwJ4Uve+rUfTU/7KF+hqey6eBw4FNym+FdgSeb2GT/Snew58AxlG+DyJiaeB2it+jlSh+P34VERtk5kXlemeWP8PdMvNA4J8U324tk5lnRkQvit+308rn9h3guiiG5zQ4sHy+y1L83AEeL18jSa1k4JbU4CWKP7qtlpn3AnsBG1H84Z4eNWNtM/OuzJyQmXMyczxFYP1cufn1wMCaHrv9gVGZOauZw43KzDGZ+R5FmBhYtu8MTMrMUeWy84F/12w3myIkrZaZ72Tmfc3sf0VganPPNSI+CWwNfK/czzjgNxSBpMG9mXlbWcc1QE/gjMycTREs+0TECjXrX56ZEzPzTeBEYEjNa/fbzJxZvh4nAxtGxPI1296Ymfdn5hyKsNkzM0/JzHfLse8XA0PLdYcAp5ffQLzI3MG+NfYHfpuZj5T1HA9sES2PbX+H4gNBc73cw4BbM/NVitD4xYhYaZ51Hil78B+n6FX90FjvGisAM+dpm03xvBte/08A55Wv6yRgEjCg5jmekpkvZ+Y0ig93tT/bOcAPM3NWOWzl68AJmTml5mf05Zj/cJP3gSWBvhGxRGY+n5nPtLD+zZl5T3mMEyhe908CuwLPZ+bvym9jHgGuo/gg2FoHUAzDuaX8Hb0dGEvxO9Xg0sycVB5jdtk2k+L1ltRKBm5JDXoBryzoRpl5a2buRhHW96AYS9swnGOziPhrOcxhBnAoReihHHZyMx+EwqF8uBe3Vm2IfgtYpry/GtDYK52ZSdEL3eC7QABjImJSRDQ3Dng6Re9tc1YDXmkYLlN6geJ1a/CfmvtvA//NzPdrHlNTN7V1l/tagmIIQdeIOKMcrvA6H/SAfqKZbdcAViuHBbwWEa8B3wdWrql93mMtiNVqt8nMNyher17NblG4GFg5IuYaLhIRPYDBlD/vzHyAovd1v3m234ji9doH2AxYuoVjvUrRC1trehOv/7w/o9r3Ue3r8kLZ1mBaZr5T83gN4Pqa1/txijC9Mi0oh8ccRRHQX46IK+czPKf2vf0Gxe/oauXxN5vnZ74/RU98a60BDJ5nH1sz9+9BU9/4LAu8tgDHkTo9A7ckImITivDUXO/vfJU9ZHcCfwEaxoKPBG4CPpmZywO/pgi/Da4A9o2ILYAewF8X4tBTKYYmAMXJXbWPM/Pfmfm1zFyNolfyV9H0DBx3AJtGRO8mlkH5DUDMPQPL6sC/FqLmBp+cZ1+zgf9SBM89KIa5LA/0Kdepfe2y5v6LwHOZuULNbdnMbOipnNrEsRbESxThrCiiGM6wIvN57mWP6I8ohkTU1v4lYDmKn8W/y3HRvWhiWEkWrgYeoBgW0pzxwLqtejZNm+s5UrxGL9WWMs/6LwJfnOc1756Z830/ZObIzNy6PF4C/9fC6o0/t4hYhuKD7Uvl8e+e5/jLZOZhzdTb3HO4fJ59LJ2ZZ7SwDRQnRz/a4pOUNBcDt9SJRcRyEbErxdftf8jMCTWLu0ZE95rb/zSx/R4RMTQiPhaFTSmGjDxYrrIsRa/wO+WyeXswb6EIHacAV5XDIxbUzUD/KE6I6wZ8k5pevogYXBOiX6UIEO/Pu5PMvINiTOz1EbFxRHSLiGWjmDLxkHIoxt+An5SvxwDgK7TcKz8/B0RE34hYiuI1uLbskV0WmEXRi7wUxUwwLRkDvF6ejNej7CHvV36QgmKM9PHlz6k38K357O8/fHCCHBQfnP5fRAwsx+n/GPh7Zj7fiud4OcUQitq5m4cBv6UYCjOwvG1FMcSofzP7OQMYHhHN9eDewgfDlRbGFcAPIqJneV7AScAfWlj/18DpDUOiyu32mN9BIuLTEfH58nV8h6KX/UPvxxo7R8TW5e/fqRSv+4vAaIox8gdGxBLlbZOIWL/cbt6fYVNtfwB2i4gdy/dM9yhO8GzuQ2eDz1GcYyCplQzcUuf0x4iYSdHDdQLwU+D/zbPOcRRhoOH2lyb28yrwNeBp4HWKP+BnZWZDCP0GcEp5rJOYZ8rAclzqKIqe3JEL80Qy878UwxPOpAiofSnGoTaMBd8E+HtEvEHR235kZj7XzO6+TBHcrgJmABOBQRS931CcmNaHoofxeooxvbcvTN2ly4FLKYbLdKc4EQ/gMoohDf8CHuODDzBNKkP6bhTB9TmKXvLfUPSOQ9HL/EK57M/lcVtyMvD7cpjBkPKbixMpxghPpTj5cmgL289b2w8pzw8oT9TbHji3/Pah4fYw8CeaOaGz/DB4N3BsM4e6jCKc9mhNXU04jeJ
"text/plain": [
"<Figure size 864x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"_ = firefox_bump_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"DL Savings Compared to NAR (%)\",title=\"DL Savings Compared to NAR (more is better)\", xlabel=\"\", ylabel=\"Savings in %\")"
]
},
{
"cell_type": "markdown",
"id": "76dc2641-3a15-48d2-9334-f4e3c2539f1e",
"metadata": {},
"source": [
"As expected, we don't see any gains (~1%) here."
]
},
{
"cell_type": "markdown",
"id": "6eb4bd07-0430-4e0e-813a-30fcb8a2e375",
"metadata": {},
"source": [
"## 3. Unstable to Stable Channel Jump\n",
"\n",
"In this scenario, we're going to simulate a stable -> unstable jump for the same NixOS machine configuration we used in the mass rebuild simulation."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "2a0cb128-24bf-4714-ab8d-aab8f6cfd291",
"metadata": {},
"outputs": [
{
"data": {
2022-03-08 15:26:17 +01:00
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
" <td>387.457979</td>\n",
" <td>375.167475</td>\n",
" <td>12.290504</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
" <td>610.885612</td>\n",
" <td>310.540811</td>\n",
" <td>300.344801</td>\n",
" <td>17.226084</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
" <td>1658.069335</td>\n",
" <td>861.781638</td>\n",
" <td>817.549735</td>\n",
" <td>-129.705850</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
" <td>478.120248</td>\n",
" <td>307.478671</td>\n",
" <td>170.641577</td>\n",
" <td>18.042290</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
"0 NAR 387.457979 375.167475 \n",
"1 Casync 610.885612 310.540811 \n",
"2 File 1658.069335 861.781638 \n",
"3 Compressed File 478.120248 307.478671 \n",
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
"0 12.290504 0.000000 \n",
"1 300.344801 17.226084 \n",
"2 817.549735 -129.705850 \n",
"3 170.641577 18.042290 "
2022-03-04 19:28:08 +01:00
]
},
2022-03-08 15:26:17 +01:00
"execution_count": 18,
2022-03-04 19:28:08 +01:00
"metadata": {},
2022-03-08 15:26:17 +01:00
"output_type": "execute_result"
2022-03-04 19:28:08 +01:00
}
],
"source": [
2022-03-08 15:26:17 +01:00
"channel_jump_results = analyse_benchmark_results(b[\"channelJump\"])\n",
"channel_jump_results"
2022-03-04 19:28:08 +01:00
]
},
{
2022-03-08 15:26:17 +01:00
"cell_type": "code",
"execution_count": 14,
"id": "9c1ff374-911d-4f9b-bf62-cb09cdbdf317",
2022-03-04 19:28:08 +01:00
"metadata": {},
2022-03-08 15:26:17 +01:00
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtQAAAGGCAYAAABbgxaRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwnUlEQVR4nO3deZgdZZn38e9NAiQsYggBgTBJWGQPAUNwhGBYBAQMOMMOURBBGEQQHXckOIOALyLK6EBEBQSDGUBFFlmH1REIgoGwSzCEBBIihD1mud8/qjqcNN2dDtWnTy/fz3X11af2u6qru3/nOU9VRWYiSZIk6b1ZodEFSJIkSd2ZgVqSJEmqwEAtSZIkVWCgliRJkiowUEuSJEkVGKglSZKkCgzUUgNFxJiImNHoOnqKiDgyIu6u07ozIjZuZdo6EXFnRLwWEd+v0/Yvjoj/rMe6W9hW3Y5jFZ15DKqKiDMj4uTydaf+nkfE1IgYs5zLdLm/RRExNiKuaHQdUnsYqKWKIuLGiPhOC+P3i4gXIqJvI+paHlX/mZYBbFFEvF5+TYuIX0TEBzuyzi7sWOAl4H2Z+aWqK+uMQBsRe9a8CZgTEXdExNh6brOeulIgjIhBwKeACxux/czcMjNv76zttXTsI2J8RFxWZb2ZeQ2wVUQMr1Sg1AkM1FJ1FwPjIiKajR8HXJ6ZCzu/pIb4v8xcDVgD2B14C3ggIrZqbFmdYgjwaL6HJ2U14g1XRBwA/A9wKTAYWAf4NvCJzq6lhzoSuD4z32p0Id1Vze/FRIo3rFKXZqCWqvstsCYwumlERAwA9gUujYiVI+K8iJhZfp0XESu3tKLm3QpqP+JuagWKiK9ExOyImBUR+0fE3hHxZET8PSK+UbPsChHxtYj4a0TMjYhJEbFmC9tcFbgBWK+mhXm95am7VmYuysy/Zua/AXcA42u2Nbb8OPqViLg9IjYvxx8VEb+vme/piJhUM/xcRIyoOUbHRcRTEfFyRPy4hTczTct9JCLuj4h55feP1Ew7KiIeK1ton4mIzzVb9t/LYzwzIj7T2v5GxMXAp4GvlMdu97aOXc3P8asR8QLwi2br2xy4APjncn2v1EweEBHXlTXfGxEb1Sy3WUTcXJ4HT0TEQa3UG8C5wH9k5kWZOS8zF2fmHZl5TLN5zymP8bSI+Hh7jl3N/n2p5jw9qvZ4lT+zSvuxLBHxbETsXjO8pMU0IoaW59FR5bn1cnlObR8RU8rz879qlj0yIu6JiPPLc+nxiNitjc1/nOLcb6229SLiqig+GZgWEV+omTYqIiZHxKsR8WJEnFuO7xcRl0Xxu/xKeT6vs6x9b219bdT2jYh4qVzH4TXjVy7Ph+nlei6IiP7R8t+Pw4BvAAeXw38p17FGRPysPCeej4j/jIg+zY7xDyLi77zzd+N2YJ+2apa6AgO1VFHZCjWJ4iPeJgcBj2fmX4BvAh8GRgDbAKOAb73HzX0A6AesT9Gi+FPgCOBDFIH+2xGxYTnvF4D9gY8C6wEvAz9uof43KALAzMxcrfya2UF1X13WRRTdPyYCJwODgOuB30fEShThY3QUbwLWBVYEdiyX2xBYDZhSs959ge3Lug4C9my+4SjePFwH/AgYSBEir4uIgeUss8v1vA84CvhBRGxXLrsX8GXgY8AmFC3uLcrMI4HLge+Vx+4Wln3sPkDxJmwIzVrfMvMx4DjKFv/MfH/N5EOB04EBwNPAGWW9qwI3A78C1i7n+0lEbNlCyZsCGwBXtrZPpR2AJ4C1gO8BPyvDOLRx7Gr2bw2K8/Ro4MdRvMnsyP3oCDtQ/HwPBs6j+LntDmwJHBQRH2027zMUx+M04Opo4Q1qaWuKY/cuEbEC8HvgLxTHZzfg5IhoOod/CPwwM98HbETxtwWKN21rUPzsBlKcI+1pAW9tfS35QLl/65fbmxARm5bTzgY+SHFOb1zO8+1W/n78Cvgu8OtyeJtyHZcAC8vltwX2AD5bs/2mY7w25TkBPAYMjYj3tWNfpYYxUEsd4xLgwIjoXw5/qhwHcDjwncycnZlzKILEuPe4nQXAGZm5ALiC4p/fDzPztcycCkwFmvobfg74ZmbOyMz5FC0+B0T7uxh0RN0zKYIjFKHlusy8uaz/HKA/8JHMfAZ4jeKf9UeBG4HnI2KzcviuzFxcs96zMvOVzJwO/G+5XHP7AE9l5i8zc2FmTgQep+zWkJnXlS3pmZl3ADfxzqcMBwG/yMxHysAwfjn3e1nHbjFwWmbOX85uAVdn5n1lN6LLeWe/9wWezcxflPv6Z+Aq4IAW1tH0hmLWMrb1t8z8aWYuojiX16XoGrKsYwfFefqdzFyQmdcDr1ME+Y7cj47wH5n5dmbeBLwBTCx/Zs8Dd1GEviazgfPKffo1RWBureX0/RTnc0u2BwZl5ncy8x/luf9T4JBy+gJg44hYKzNfz8w/1YwfCGxcfgr0QGa+2o59bG19rTm1PC/voHhDelD5RuoY4IuZ+ffMfI0iMB/S1opqla3pHwdOzsw3MnM28INm65iZmeeXP/um34um4/j+9m5LaoQuf7GU1B1k5t0RMQfYLyLuo/in+S/l5PWAv9XM/rdy3Hsxtww48E7r1Is109+iaM2FovXzNxFRG0QXUYSi59uxrY6oe33g7y2tLzMXR8Rz5TxQtFKPoWi9ugN4hSJM/zPv/vj8hZrXb/LOPrdVf9M+rA8QRReG0yha3VYAVgEerln2gWbLLY9lHbs5mfn2cq4TWt/vIcAOsXT3kL7AL1tYx9zy+7rAtPZsKzPfLBunV4NlHjsoztPaawea/4w6Yj86QvPfndZ+lwCeb9ZHvq3fh5eB1VuZNoSie8QrNeP6UAR4KFr0vwM8HhHTgNMz81qKY7ABcEVEvB+4jOIN84JWttOktfW1WHf5BrJJ0z4OovgZP/DOhxREWXd7DaH45GlWzTpWAJ6rmee55gvxznF8ZTm2JXU6W6iljnMpRcv0OOCmzGz65zyT4p9Jk38qx7XkTYp/XE0+UKGe54CPZ+b7a776la1vzbV0Md3y1N2aT/JOUFhqfWWr1wa8E+6bAvXo8vUdFIH6o7TRH7UNzeuHYh+ej6I/81UUreTrlN0qrqcICVC03m7QbLkq225+7JZ18eLyXtz4HHBHs5/1apl5fAvzPlHO/6/LuQ2g6EtL28euiuXZj2V5g477XQJYv6bLC7T9+zCF4s1GS54DpjXbx9Uzc2+AzHwqMw+l6PZwNnBlRKxatoyfnplbAB+haM3/VCvbWKK19bUy+4Bm05r28SWKNxhb1tS8RhYXIUPL52vzcc8B84G1atbxvszcso1lADan+NSiPa3xUsMYqKWOcylF/8tjeKe7BxT9hr8VEYMiYi2Kvs+t3U7qIeCwiOhT9uP9aCvztccFwBkRMQSKW3lFxH6tzPsiMDAi1niPdS9R1j4sIs6nCMinl5MmAftExG4RsSLwJYp/sH8sp98B7AL0z8wZFEF8L4qPuR9s707XuB74YEQcFhF9I+JgYAvgWmAlYGVgDrCwbHHdo2bZScCREbFFRKxC0Rq7PN7TsavxIjC47F/eHtdS7Ou4iFix/No+yos+a5WtrKcAp0ZxUd77yr7rO0XEhHZsa1nHrop270c7PAQcUq5jJNW7jawNfKFc34EUQe/6Vua9ntZ/d+8DXo3iotT+5e/LVhGxPUBEHBERg8ouTq+UyyyKiF0iYuvyIr5XKbpyLGpxCzVaW18bi5weEStFxGiK0P4/5bI/pegrv3a53vVr+n239PfjRYq+zysAZOYsiq5B36855zZq1k+9JR+luOhR6tIM1FIHycxnKcLhqsA1NZP+E5hM0Wr1MPDnclxLTqLo4/sKRT/c31Yo6YdlHTdFxGvAnygu+mmp9scpQuAzUdxBYL3lrBvKu1JQ/LO/neKCte0z8+FyG09QXEB5PkWL1yeAT2TmP8rpT1L0tb2rHH6V4gKle2q6ubRbZs6lCARfoujm8BVg38x8qewD+gWK4PwycBg1P7PMvIHiIrXbKC6au205N7+8x6652yj6w78QES8ta+Zyf/ag6I86k6JLxdkUwbel+a+k6NP
"text/plain": [
"<Figure size 864x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
2022-03-04 19:28:08 +01:00
"source": [
2022-03-08 15:26:17 +01:00
"_ = channel_jump_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"Downloaded Size (MB)\",title=\"Volume to Download for the Channel Jump (less is better)\", xlabel=\"\", ylabel=\"Size in MB\")"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 15,
"id": "4dcd7cc8-1185-411b-8e30-115f5c5f516b",
2022-03-04 19:28:08 +01:00
"metadata": {},
"outputs": [
{
"data": {
2022-03-08 15:26:17 +01:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtwAAAGGCAYAAABIVFZlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2lklEQVR4nO3debxd873/8dcniUrMrVmCoFpCQglqqBpLzdcYlLha+VVpKTXVUJTWVVx6dZCWalUQNaWGGmssIjQkMVQM1ZASQ9KYIuTz+2OtEzuxz8lJZGWfnP16Ph7nkb3X+NnDyXnv7/6stSIzkSRJklSNLo0uQJIkSerMDNySJElShQzckiRJUoUM3JIkSVKFDNySJElShQzckiRJUoUM3JI6pIh4OyJWbXQdnVFEbBER4xpdx9wWET+NiCMbXcfMIuIrEfHMHKx3aUScUUVNcyoiro2I7RtdhzS/MXBLTSYiXoyI9yJickRMjIi/RcS3I6JLzTLt/kMfEd+MiKfL7b0aETdFxKKfts7MXCQzn/+025kdEfGZiDg1Ip6NiHfK5+qSiOg9L+totIjIiPj8HK7bu1z/ppmm/zEiTp1p2ioRMS0iftlKDe+UH7xejojzIqJrG/tdGjgQuGhO6q5SZt6XmV+cl/us9ztcvp+3+ZSbPgs481NuQ2o6Bm6pOe2cmYsCK1P8AT0OuHh2NxIRXwV+Auxbbm9NYOjcLHQe+xOwC7AfsDiwDvAosHUji5qVtoJoA305IjadxTIHAm8BAyJiwTrz18nMRYCvAvsAB7exrYOAmzPzvTkpdnZERLeq99HRRKFLZg4HFouI/o2uSZqfGLilJpaZkzJzGEWYGRgRa8/mJjYAHszMv5fbezMzf5+ZkwEiYseI+HtE/Cci/lU7whkRf4mIw2s3FhGPR8Tu5e3po6zlaN0vytHzyRHxcESsVrPe1yLimYiYFBG/jIh7IuJb5bzPl/cnRcTrEXFVvQdSjvxtC+yamY9k5ofl8/OLzLy4XGaFiBgWEW9GxNiIOKRm/VMj4upyJHdyRIyKiC9ExAkR8Vr5+L9Ws/zdZQvE8LK2GyLiczXzr46If5fz7o2ItWrmXRoRv4qImyPiHWDLsrZrImJCRLwQEd+rWb5Huc5bEfFk+brVFRH3ljcfL0eX9ymnH1I+5jfL52CF1rZROhuY1bckBwInAVOBnVtbKDPHAg8A67axra8D99Q8ji0iYlxEHFs+/+MjYreI2CEi/lE+jh/WLL9gRJwfEa+UP+e3fAio2dZxEfFv4HcR0SUijo+I5yLijYgYWvv61YqZWnjK7bxcvk+eiYi2PtAtFRG3l8veExEr12xnjXLem+V29i6nDwL2B44tX8M/R8RlwErAn8tpx5bLfjmKb7kmRvH7t0XN9u+OiDMj4gHgXaClxetuYMc2apY0EwO3JMpRq3HAV2Zz1YeB7SLitIjYND45SvkORahaguIP9KERsVs5bwiwb8uCEdGHYsT9JurbFzgN+CwwlvJr7YhYimJk+gRgSeAZYJOa9X4M3Fau1wv4v1a2vw0wPDP/1cbjvYLieVoB2BP4yUxhaWfgsnJffwdupfh/tidwOp9sdziQYtR2BeBD4Oc1824BVgeWAR4DLp9p3f0onoNFgb8BfwYeL/e1NXBkRGxXLvsjYLXyZztgYGsPMDM3L2+uU7b1XBURWwE/BfYGlgf+CVzZ2jZKvwC+EK20METEVyhejyspvhU5sLUNRcQaFO/NsW3sry/Fa19rOaA7xXNyCvAb4BvA+uX2TomPjxM4EfgyRahfB9iQ4sNA7bY+R/EeHQR8D9iNYvR9BYqR+l+0UV/LY/kicDiwQfmt0HbAi22ssj/Fe3gpYCTl+yAiFgZup/g9Wobi9+OXEbFWZg4ulzu7fA13zswDgJcovt1aJDPPjoieFL9vZ5SP7QfANVG057Q4oHy8i1K87gBPlc+RpHYycEtq8QrFH912y8z7gN2B9Sj+cL8RNb22mXl3Zo7KzGmZ+QRFYP1qufp1wLo1I3b7A9dm5pRWdndtZg7PzA8pwsS65fQdgDGZeW057+fAv2vWm0oRklbIzPcz8/5Wtr8kML61xxoRKwKbAceV2xkJ/JYikLS4LzNvLeu4GlgaOCszp1IEy94RsUTN8pdl5ujMfAc4Gdi75rm7JDMnl8/HqcA6EbF4zbo3ZOYDmTmNImwunZmnZ+YHZe/7b4AB5bJ7A2eW30D8ixmDfXvsD1ySmY+V9ZwAbBxt97a/T/GBoLVR7oHALZn5FkVo/HpELDPTMo+VI/hPUYyqfqLXu8YSwOSZpk2leNwtz/9SwAXl8zoGGAP0q3mMp2fma5k5geLDXe1rOw34UWZOKdtW/h9wYmaOq3mN9oxZt5t8BCwI9ImIBTLzxcx8ro3lb8rMe8t9nEjxvK8I7AS8mJm/K7+NeQy4huKDYHt9g6IN5+byd/R2YATF71SLSzNzTLmPqeW0yRTPt6R2MnBLatETeHN2V8rMWzJzZ4qwvitFL21LO8dGEfHXss1hEvBtitBD2XZyEx+HwgF8chS3Vm2IfhdYpLy9AjB9VDozk2IUusWxQADDI2JMRLTWB/wGxehta1YA3mxplyn9k+J5a/Fqze33gNcz86Oa+9TUTW3d5bYWoGgh6BoRZ5XtCv/h4xHQpVpZd2VghbItYGJETAR+CCxbU/vM+5odK9Suk5lvUzxfPVtdo/AbYNmImKFdJCJ6AHtRvt6Z+SDF6Ot+M62/HsXztQ+wEbBwG/t6i2IUttYbdZ7/mV+j2vdR7fPyz3JaiwmZ+X7N/ZWB62qe76cowvSytKFsjzmSIqC/FhFXzqI9p/a9/TbF7+gK5f43muk1359iJL69Vgb2mmkbmzHj70G9b3wWBSbOxn6kpmfglkREbEARnlob/Z2lcoTsTuAuoKUXfAgwDFgxMxcHfk0RfltcAewbERsDPYC/zsGux1O0JgDFwV219zPz35l5SGauQDEq+cuofwaOO4ANI6JXnXlQfgMQM56BZSXg5TmoucWKM21rKvA6RfDclaLNZXGgd7lM7XOXNbf/BbyQmUvU/CyamS0jlePr7Gt2vEIRzooiinaGJZnFYy9HRE+jaImorf2/gMUoXot/l33RPanTVpKFocCDFG0hrXkC+EK7Hk19MzxGiufoldpSZlr+X8DXZ3rOu2fmLN8PmTkkMzcr95fA/7Sx+PTXLSIWofhg+0q5/3tm2v8imXloK/W29hgum2kbC2fmWW2sA8XB0Y+3+SAlzcDALTWxiFgsInai+Lr9j5k5qmZ214joXvPzmTrr7xoRAyLis1HYkKJl5KFykUUpRoXfL+fNPIJ5M0XoOB24qmyPmF03AX2jOCCuG3AYNaN8EbFXTYh+iyJAfDTzRjLzDoqe2OsiYv2I6BYRi0ZxysSDy1aMvwE/LZ+PfsA3aXtUfla+ERF9ImIhiufgT+WI7KLAFIpR5IUozgTTluHAf8qD8XqUI+Rrlx+koOiRPqF8nXoB353F9l7l4wPkoPjg9N8RsW7Zp/8T4OHMfLEdj/EyihaK2nM3DwQuoWiFWbf82ZSixahvK9s5CxgUEa2N4N7Mx+1Kc+IK4KSIWLo8LuAU4I9tLP9r4MyWlqhyvV1ntZOI+GJEbFU+j+9TjLJ/4v1YY4eI2Kz8/fsxxfP+L+BGih75AyJigfJng4hYs1xv5tew3rQ/AjtHxHble6Z7FAd4tvahs8VXKY4xkNROBm6pOf05IiZTjHCdCJwH/PdMyxxPEQZafu6qs523gEOAZ4H/UPwB/1lmtoTQ7wCnl/s6hZlOGVj2pV5LMZI7ZE4eSGa+TtGecDZFQO1D0Yfa0gu+AfBwRLxNMdp+RGa+0Mrm9qQIblcBk4DRQH+K0W8oDkzrTTHCeB1FT+/tc1J36TLgUop2me4UB+IB/IGipeFl4Ek+/gBTVxnSd6YIri9QjJL/lmJ0HIpR5n+W824r99uWU4Hfl20Ge5ffXJxM0SM8nuLgywFtrD9zbT+iPD6gPFBva+D88tuHlp9Hgb/QygGd5YfBe4BjWtnVHyjCaY/21FXHGRTvmyeAURQHqrZ
2022-03-04 19:28:08 +01:00
"text/plain": [
2022-03-08 15:26:17 +01:00
"<Figure size 864x360 with 1 Axes>"
2022-03-04 19:28:08 +01:00
]
},
2022-03-08 15:26:17 +01:00
"metadata": {
"needs_background": "light"
},
2022-03-04 19:28:08 +01:00
"output_type": "display_data"
}
],
"source": [
2022-03-08 15:26:17 +01:00
"_ = channel_jump_results.plot.bar(figsize=(12,5), x=\"Name\",y=\"DL Savings Compared to NAR (%)\",title=\"DL Savings Compared to NAR (more is better)\", xlabel=\"\", ylabel=\"Savings in %\")"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "markdown",
2022-03-08 15:26:17 +01:00
"id": "b75a48a8-e66b-46b7-97eb-802a2a674d04",
2022-03-04 19:28:08 +01:00
"metadata": {},
"source": [
2022-03-08 15:26:17 +01:00
"I'm surprised on this one! I did not expect any gains, yet here we are, looking at a 18% improvement for compressed files, 17.2% improvement for Casync. While not amazing, it's big enough to be noticeable."
]
},
{
"cell_type": "markdown",
"id": "1290125c-ad85-47ab-974f-8040113e6ba0",
"metadata": {},
"source": [
"# Gimp Bump"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 20,
"id": "043cca46-d47c-41af-90de-576fbb73ce9e",
2022-03-04 19:28:08 +01:00
"metadata": {},
"outputs": [
{
"data": {
2022-03-08 15:26:17 +01:00
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
" <td>247.506391</td>\n",
" <td>20.270771</td>\n",
" <td>227.235619</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
" <td>363.372208</td>\n",
" <td>13.057223</td>\n",
" <td>350.314985</td>\n",
" <td>35.585959</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
" <td>1127.324116</td>\n",
" <td>38.239911</td>\n",
" <td>1121.558323</td>\n",
" <td>-88.645563</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
" <td>300.300404</td>\n",
" <td>10.187397</td>\n",
" <td>290.113007</td>\n",
" <td>49.743419</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
2022-03-04 19:28:08 +01:00
"text/plain": [
2022-03-08 15:26:17 +01:00
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
"0 NAR 247.506391 20.270771 \n",
"1 Casync 363.372208 13.057223 \n",
"2 File 1127.324116 38.239911 \n",
"3 Compressed File 300.300404 10.187397 \n",
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
"0 227.235619 0.000000 \n",
"1 350.314985 35.585959 \n",
"2 1121.558323 -88.645563 \n",
"3 290.113007 49.743419 "
2022-03-04 19:28:08 +01:00
]
},
2022-03-08 15:26:17 +01:00
"execution_count": 20,
2022-03-04 19:28:08 +01:00
"metadata": {},
2022-03-08 15:26:17 +01:00
"output_type": "execute_result"
2022-03-04 19:28:08 +01:00
}
],
"source": [
2022-03-08 15:26:17 +01:00
"gimp_bump_results = analyse_benchmark_results(b[\"gimpBump\"])\n",
"gimp_bump_results"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "markdown",
2022-03-08 15:26:17 +01:00
"id": "f98c3d4d-939b-457a-8d52-9cdf91a81867",
2022-03-04 19:28:08 +01:00
"metadata": {},
"source": [
2022-03-08 15:26:17 +01:00
"# Emacs Bump"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 22,
"id": "da6473fa-7ab4-4648-b5f8-389bc994b6b0",
2022-03-04 19:28:08 +01:00
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
2022-03-08 15:26:17 +01:00
" <td>110.330516</td>\n",
" <td>39.744307</td>\n",
" <td>70.586210</td>\n",
2022-03-04 19:28:08 +01:00
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
2022-03-08 15:26:17 +01:00
" <td>177.733558</td>\n",
" <td>44.726448</td>\n",
" <td>133.007110</td>\n",
" <td>-12.535484</td>\n",
2022-03-04 19:28:08 +01:00
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
2022-03-08 15:26:17 +01:00
" <td>462.193035</td>\n",
" <td>115.231532</td>\n",
" <td>350.550795</td>\n",
" <td>-189.932172</td>\n",
2022-03-04 19:28:08 +01:00
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
2022-03-08 15:26:17 +01:00
" <td>136.599691</td>\n",
" <td>40.735590</td>\n",
" <td>95.864101</td>\n",
" <td>-2.494152</td>\n",
2022-03-04 19:28:08 +01:00
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
2022-03-08 15:26:17 +01:00
"0 NAR 110.330516 39.744307 \n",
"1 Casync 177.733558 44.726448 \n",
"2 File 462.193035 115.231532 \n",
"3 Compressed File 136.599691 40.735590 \n",
2022-03-04 19:28:08 +01:00
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
2022-03-08 15:26:17 +01:00
"0 70.586210 0.000000 \n",
"1 133.007110 -12.535484 \n",
"2 350.550795 -189.932172 \n",
"3 95.864101 -2.494152 "
2022-03-04 19:28:08 +01:00
]
},
2022-03-08 15:26:17 +01:00
"execution_count": 22,
2022-03-04 19:28:08 +01:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2022-03-08 15:26:17 +01:00
"emacs_bump_results = analyse_benchmark_results(b[\"emacsBump\"])\n",
"emacs_bump_results"
2022-03-04 19:28:08 +01:00
]
},
{
2022-03-08 15:26:17 +01:00
"cell_type": "markdown",
"id": "5c603d08-b887-4249-9a63-2614a3c00212",
2022-03-04 19:28:08 +01:00
"metadata": {},
"source": [
2022-03-08 15:26:17 +01:00
"# OpenMPI Bump"
2022-03-04 19:28:08 +01:00
]
},
{
"cell_type": "code",
2022-03-08 15:26:17 +01:00
"execution_count": 24,
"id": "3efbf4a8-0637-4fa1-be31-2450a49dc249",
2022-03-04 19:28:08 +01:00
"metadata": {},
"outputs": [
{
"data": {
2022-03-08 15:26:17 +01:00
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Name</th>\n",
" <th>Closure Size (MB)</th>\n",
" <th>Downloaded Size (MB)</th>\n",
" <th>Re-used Size (MB)</th>\n",
" <th>DL Savings Compared to NAR (%)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NAR</td>\n",
" <td>139.540192</td>\n",
" <td>3.335962</td>\n",
" <td>136.204230</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Casync</td>\n",
" <td>214.708487</td>\n",
" <td>4.380815</td>\n",
" <td>210.327671</td>\n",
" <td>-31.320878</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>File</td>\n",
" <td>636.211984</td>\n",
" <td>10.100060</td>\n",
" <td>632.297983</td>\n",
" <td>-202.762988</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Compressed File</td>\n",
" <td>167.518237</td>\n",
" <td>3.260960</td>\n",
" <td>164.257277</td>\n",
" <td>2.248310</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
2022-03-04 19:28:08 +01:00
"text/plain": [
2022-03-08 15:26:17 +01:00
" Name Closure Size (MB) Downloaded Size (MB) \\\n",
"0 NAR 139.540192 3.335962 \n",
"1 Casync 214.708487 4.380815 \n",
"2 File 636.211984 10.100060 \n",
"3 Compressed File 167.518237 3.260960 \n",
"\n",
" Re-used Size (MB) DL Savings Compared to NAR (%) \n",
"0 136.204230 0.000000 \n",
"1 210.327671 -31.320878 \n",
"2 632.297983 -202.762988 \n",
"3 164.257277 2.248310 "
2022-03-04 19:28:08 +01:00
]
},
2022-03-08 15:26:17 +01:00
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
2022-03-04 19:28:08 +01:00
}
],
"source": [
2022-03-08 15:26:17 +01:00
"openmpi_bump_results = analyse_benchmark_results(b[\"openmpiBump\"])\n",
"openmpi_bump_results"
2022-03-04 19:28:08 +01:00
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python3 - python",
"language": "python",
"name": "ipython_python"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
},
"toc-showtags": false
},
"nbformat": 4,
"nbformat_minor": 5
}
