initial commit after completion

config.zsh

@@ -0,0 +1,34 @@
+# Writedown
+# generic configuration defaults for rendered files
+
+WRITEDOWN_TITLE="Integration and deployment of the DECODE OS and HUB platform"
+WRITEDOWN_AUTHOR="Denis Roio, Ivan Jelincic"
+WRITEDOWN_AFFILIATION="Dyne.org Foundation"
+WRITEDOWN_DATE="Tuesday 12 June 2018"
+WRITEDOWN_TAGS="[gnu, linux, embedded]"
+
+# number for each section
+WRITEDOWN_NRSEC=yes
+# table of contents
+WRITEDOWN_TOC=yes
+# bibliographic citation style (see writedown/citstyle)
+WRITEDOWN_CITSTYLE=harvard-kings-college-london
+# font size
+WRITEDOWN_FONTSIZE=14pt
+# bibtex file for bibliographic sources
+# WRITEDOWN_BIB=views/references.bib
+# latex template (header and footer)
+# WRITEDOWN_LATEX_TEMPLATE=views/template.tex
+
+# different formats as supported by pandoc.
+# to activate uncomment and fill, then use dash (-) as first argument
+# i.e: ./writedown/render -
+# WRITEDOWN_OUTPUT_FORMAT=epub
+# WRITEDOWN_OUTPUT_EXTENSION=epub
+
+# default pandoc base command
+# WRITEDOWN_PANDOC="pandoc --smart --standalone -f markdown
+
+# Experimental features are commented below
+
+# WRITEDOWN_ZOTERO="no"

views/build.md

@@ -0,0 +1,53 @@
+DECODE OS builds and hardware integration
+=========================================
+
+The DECODE operating system is built in a number of stages: this process is detailed in D4.4 and will be briefly documented here.
+
+This section outlines how to build DECODE OS for our DECODE HUB hardware target architecture which is the hard-float ARM (armhf). For the purpose we adopted the Devuan Ascii (2.0.0) armhf port.
+
+Each build stages illustrated below is independent from each other and can be triggered on its own; however the final stage requires the results from the previous ones to be able to integrate them together.
+
+
+Userspace
+---------
+
+The first stage built is the actual userspace, where the system is bootstraped from scratch, and packages and configurations are added on top of it. This is the step most extensively described in D4.4. The tools we use to build our userspace are made available in the [](https://github.com/DECODEproject/decode-os) git repository. These tools are a higher-level abstraction on top of Devuan's Arm-SDK, which is the way ARM-based images are created in Devuan. The port of Devuan to ARM has gathered an active community that can be also reached on the Freenode IRC channel #devuan-arm.
+
+The process of building via the Devuan SDK is abstracted using a concept called _blends_: it offers us a more high-level approach to configuring a base system after it has been bootstrapped. Using this _blend_, we are able to easily and reproducibly install various software - from the init system (OpenRC), to the very final userspace tools needed for DECODE like Chainspace and the JVM, tor-dam and Golang for more applications.
+
+Along with installing software, the blends allow us to provide file overlays, which enable us to have a directory structure that resembles the bootstrapped system, and use the overlay to reconfigure any existing files. Finally, it also gives us a space where we can mark certain binaries with needed flags that are used in the kernelspace.
+
+
+Kernelspace
+-----------
+
+The ARM kernelspace consists of mainline Linux (official kernel.org) releases (version 4.9.y) which in the special case of DECODE OS requirements for integrity and security of the system we patch with the unofficial grsecurity patch that is maintained for this very kernel version. It is maintained at [](https://github.com/minipli/linux-unofficial_grsec/releases) and (unlike x86) does not suffer from the recent Meltdown and Spectre attacks (CVE-2017-5715, CVE-2017-5753, and CVE-2017-5754) while this does not apply for _all_ ARM processors.
+
+The tools used for building this stage are packed into a single git repository which is a suite of makefiles that contain deterministic rules for building the Linux kernel, the _initramfs_, and finally packs the results into a tarball, and a ready-to-flash raw image - both of which can be used and put onto a microSD card to be used with the Olimex OLinuXino LIME2. This suite can be found at the DECODE project's Github: [](https://github.com/DECODEproject/roundshot/)
+
+Roundshot is in fact a new software product part of this DECODE deliverable, to provide a seamless and minimalist way to distribute signed updates of the DECODE OS to all running nodes, with the only need to reboot them in order to activate the updates. Roundshot is engineered with minimalism in mind and consists only of a set of declarative scripts (GNU Makefile) to pack a new system ramdisk that will operate the update.
+
+### Cross-toolchain and initramfs
+
+When ran, Roundshot will first build a musl-based GCC toolchain which can crosscompile and statically link armhf binaries for us. Static compilation is adopted in many critical parts of DECODE OS infrastructure also to keep integrity with the privilege escalation model "sup" that is described in DECODE's whitepaper.
+
+The adoption of [musl-libc](https://www.musl-libc.org/) is useful for having a static initramfs, and smaller binaries than what the GNU libc provides.
+
+The ramdisk (initramfs) consists of busybox, btrfs tools, and gnupg, which provide tools necessary for fetching our over-the-air updates to the DECODE operating system. It also contains the logic of setting up the system to boot into the actual userspace that was built in another stage.
+
+
+### The Linux kernel
+
+After finishing the initramfs, the makefile suite will continue with fetching Linux sources and the patches, applying them, and compiling the Linux kernel and the necessary device tree that ARM boards need to define their hardware. These device trees are used by the u-boot bootloader which is explained further on.
+
+### The squashfs
+
+The squashfs is the fourth phase of Roundshot, which takes the userspace we have bootstrapped and configured and compresses it into a read-only squashfs file. This very file can be rebuilt and is used when updates are shipped. If an update is available, this file is replaced in-place on the machine which has downloaded the update after rebooting.
+
+On the actual hardware, this squashfs is configured and set up through the logic found in our initramfs. The squashfs also contains the kernel modules that were built in the previous phase. The modules are not needed inside the initramfs.
+
+### The bootloader
+
+The Olimex Lime2 - much like all other ARM SoCs - uses the u-boot bootloader.  Luckily, in the case of Lime2, we don't require any proprietary blobs or firmware and we can utilize explicitly libre software here. u-boot is built using the musl-libc cross-toolchain as well, and is flashed on the microSD card. The device tree file that we compiled with Linux sources is read by u-boot at the boot stage, along with boot.scr which is a u-boot boot script which contains the necessary information of what partitions/devices the dtb, the kernel, and the initramfs can be found on.
+
+Finally, when all of this is finished, the results are packed and compressed into a tar.gz file, and a raw .img file. These files are used for initial distribution and are to be flashed on microSD cards.

views/conclusion.md

@@ -0,0 +1,16 @@
+Conclusion
+==========
+
+This brief deliverable consists of accompanying documentation to the following software implementations realised for the DECODE project:
+
+- [DECODE OS Devuan blend (ARM and VM targets)](https://github.com/DECODEproject/decode-os)
+- [Tor Dam software application (Go language)](https://github.com/DECODEproject/tor-dam)
+- [Roundshot ramdisk (GNU Makefile)](https://github.com/DECODEproject/roundshot)
+
+The implementation of the OS with the HUB hardware platform has been a straightforward process and it did not induced any compromise on the freedom of software and hardware deployed. The DECODE HUB when equipped is usable as a reliable platform for P2P networks that abide to the principles of privacy by design enounced in D4.1.
+
+The work outlined in this deliverable will proceed further to test the behavior of DECODE HUBs in various situations, functioning as edge routers to the DECODE network, with their deployement on premises of various DECODE pilot partners.
+
+The upcoming deliverable D4.10 "DECODE HUB devices piloted in trials" will report field tests conducted on pilots, while the D4.16 "Report on DECODE architecture stability and usability" shall outline the stability and usability of this component used in conjunction with DECODE application space tooling as well as standalone base for more modular P2P applications in need to deploy cloud-shaped micro-service swarms whose connections can be private, pseudonimic and encrypted.
+
+

views/deploy.md

@@ -0,0 +1,29 @@
+DECODE OS hardware deployment
+=============================
+
+This section gives a quick and visual explanation on how to deploy the provided DECODE OS image on an Olimex Lime2 board and power it up. After this the DECODE HUB will be online and automatically connect to all other DECODE nodes.
+
+Upon creating the userspace and kernelspace, we deploy our results on the Olimex OLinuXino LIME2 ARM SoC.
+
+![](01-board.jpg)
+
+Following up, we connect the SATA hard drive with the data and power
+cables that are custom-built by Olimex.
+
+![](02-sata-connected.jpg)
+
+We flash the image on a microSD card and put it in place.
+
+![](03-sdcard-inserted.jpg)
+
+The initramfs requires a network connection to be able to check for
+and eventually download available updates to the system. It assumes that
+the router it's connecting to offers DHCP.
+
+![](04-ethernet-and-power.jpg)
+
+Finally, by connecting the battery and the power supply, the board will
+start booting and if everything is done properly, in a couple of minutes
+it will boot into DECODE OS.
+
+![](05-battery.jpg)

views/hardware.md

@@ -0,0 +1,41 @@
+Hardware specifications
+=======================
+
+According to findings explicited in D4.8 by project partner Arduino, we have selected an embedded low-consumption board for its specific characteristics conforming our requirements. This choice has been made despite certain shortcomings on performance shown in D4.5 benchmarks that may regarded as negligible in this piloting phase.
+
+In brief, these characteristics are:
+- Availability of a SATA interface for storage solutions beyond SDCards
+- Free from any proprietary firmware blob for boot (100% free eligible)
+- Capable of running mainline Linux kernel without patches
+- Assembled in Europe and certified as Open Hardware
+
+DECODE HUB nodes (elsewhere indicated as "nano-nodes") are then made using the Olimex Lime2 hardware boards (based on a ARM20 processor) and connected using a gigabit Ethernet port. A LiPo battery provides stability in case of short power cuts and a noise immune power supply provides the necessary 5v energy to function.
+
+The boards are assembled in Europe and are certified Open Hardware: they do not require any proprietary blob to run. It is therefore possible to assume that all software functions running on the boards are not conditioned by the hardware manufacturer, clearing issues of liability and integrity of computations.
+
+## Manufacturer's documentation
+
+The manufacturer provides extensive documentation about these boards: 
+
+- [The Olimex Lime2 user manual](https://www.olimex.com/Products/OLinuXino/A20/A20-OLinuXIno-LIME2/resources/A20-OLinuXino-LIME2-UM.pdf)
+
+- [The full A20 hardware datasheet](https://github.com/OLIMEX/OLINUXINO/blob/master/HARDWARE/A20-PDFs/A20%20Datasheet%20v1.0%2020130227.pdf)
+
+For a quick overview of the hardware here is included the block diagram found in the  A20 datasheet (AllWinner chip):
+
+![](olimex-lime2-block-diagram.png)
+
+An illustrated image of the Lime2 board assemblage with indicated components is also reported here from its user manual. Here the front:
+
+![](olimex-lime2-front.png)
+
+And here below the rear:
+
+![](olimex-lime2-retro.png)
+
+In the following chapter is provided information on how to build the DECODE OS system to operate on this board.
+
+
+
+
+

views/index.txt

@@ -0,0 +1,6 @@
+intro.md
+hardware.md
+build.md
+deploy.md
+network.md
+conclusion.md

views/intro.md

@@ -0,0 +1,16 @@
+Introduction
+============
+
+This brief technical deliverable illustrates aspects related to the integration and deployement of the DECODE OS (D4.4) with the DECODE HUB hardware platform according to benchmarks (D4.5) and the indicated reference prototype (D4.8). This deliverable configures itself also as an operational manual about the following topics:
+
+1. Specifications of the HUB hardware adopted
+2. Procedures to build the OS for the HUB
+3. Instructions to deploy the HUB running the OS
+4. Overview of networking access and operations
+
+This document provides pilot partners in DECODE the basic knowledge to run a DECODE HUB node on their premises, assessing the impact of such a deployement within their infrastructure. As the piloting phase unfolds towards D4.10 "DECODE HUB devices piloted in trials" this document will be extended considering the security and liability implications emerged.
+
+For any enquiry, the contact person for corrections and additions to this deliverable is Denis Roio (Dyne.org) <jaromil@dyne.org>
+
+The latest version of this deliverable is made available to the public as part of the documentation of DECODE OS and attainable from its website on [](https://decodeos.dyne.org).
+

views/network.md

@@ -0,0 +1,35 @@
+Network activity
+================
+
+This section offers a brief overview of all the visible network activity that a DECODE NODE produces when operated.
+
+## Local network
+
+On bootup, DECODE nodes will request DHCP over Ethernet.
+
+In case of lack of DHCP the box will boot after a delay, but will not be online and will not join the DECODE network. The board can eventually be configured by hand by connecting a keyboard and an HDMI monitor.
+
+## Incoming traffic
+
+DECODE nodes do not require any input ports to be open of forwarded, nor any UPNP capability on the LAN gateway. Nodes route all their traffic via the Tor network and offer the possibility to application space utilities to receive any communication via Socks5 protocol router by Tor. This also means that there are no common firewall configurations that can stop the DECODE node from connecting: once it has DHCP and at least web access it will manage to join the Tor network and get in touch with all the DECODE network.
+
+## Outgoing traffic
+
+Once received a local IP via DHCP, DECODE nodes build outgoing connections to:
+
+1. pool.ntp.org (port 123) to get current time
+2. dam.decodeproject.eu (port 80) to see if there are available system updates
+3. if there are updates, dam.decodeproject.eu:80 is accessed again to download the update
+
+These is the only deterministic traffic produced by a DECODE node upon boot and can be considered a base for fingerprinting nodes running on the local network, recognising them from this initial behavior. All subsequent connections are routed through Tor and therefore opaque even to the local network administrators.
+
+## Continued operation
+
+Once this first configuration phase (initramfs phase) is done, we continue booting into the actual system. The system will again request NTP on outbound port 123 and continue by starting all the remaining services in userspace. All services are tunneled through Tor socks and therefore do not require any inbound or outbound connections outside of Tor.
+
+Tor's network requirements can be found in the Tor Project FAQ, most notably:
+
+- [Tor Outbound Ports documentation](https://www.torproject.org/docs/faq.html.en#OutboundPorts)
+- [Tor Firewall Ports documentation](https://www.torproject.org/docs/faq.html.en#FirewallPorts)
+
+DECODE nodes operate only as hidden services inside Tor and connect only to other DECODE nodes: they do not act as entry, relay or exit nodes for the Tor network.

views/template.tex

@@ -0,0 +1,177 @@
+\documentclass[a4paper]{extarticle}
+\usepackage{lmodern}
+$if(fontsize)$
+\usepackage[$fontsize$]{extsizes}
+$endif$
+\usepackage{fullpage}
+\usepackage{longtable}
+\usepackage{booktabs}
+\usepackage{amssymb,amsmath}
+\usepackage{ifxetex,ifluatex}
+\usepackage{fixltx2e} % provides \textsubscript
+\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
+  \usepackage[T1]{fontenc}
+  \usepackage[utf8x]{inputenc}
+\else % if luatex or xelatex
+  \ifxetex
+    \usepackage{mathspec}
+  \else
+    \usepackage{fontspec}
+  \fi
+  \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase}
+\fi
+% use upquote if available, for straight quotes in verbatim environments
+\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
+% use microtype if available
+\IfFileExists{microtype.sty}{%
+\usepackage{microtype}
+\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
+}{}
+\usepackage{hyperref}
+\hypersetup{unicode=true,
+            pdftitle={$title$},
+            pdfauthor={$author$},
+            $if(keywords)$
+            pdfkeywords={$for(keywords)$$keywords$$sep$; $endfor$},
+            $endif$
+            pdfborder={0 0 0},
+            breaklinks=true}
+\urlstyle{same}  % don't use monospace font for urls
+\usepackage{xcolor}
+$if(listings)$
+\usepackage{listings}
+\lstset{
+    basicstyle=\ttfamily,
+%    numbers=left,
+    numberstyle=\footnotesize,
+    stepnumber=2,
+    numbersep=5pt,
+    backgroundcolor=\color{black!10},
+    showspaces=false,
+    showstringspaces=false,
+    showtabs=false,
+    tabsize=2,
+    captionpos=b,
+    breaklines=true,
+    breakatwhitespace=true,
+    breakautoindent=true,
+    linewidth=\textwidth
+}
+$endif$
+\usepackage{color}
+\usepackage{fancyvrb}
+\newcommand{\VerbBar}{|}
+\newcommand{\VERB}{\Verb[commandchars=\\\{\}]}
+\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}}
+% Add ',fontsize=\small' for more characters per line
+\newenvironment{Shaded}{}{}
+\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.00,0.44,0.13}{\textbf{{#1}}}}
+\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.56,0.13,0.00}{{#1}}}
+\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.25,0.63,0.44}{{#1}}}
+\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.25,0.63,0.44}{{#1}}}
+\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.25,0.63,0.44}{{#1}}}
+\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.53,0.00,0.00}{{#1}}}
+\newcommand{\CharTok}[1]{\textcolor[rgb]{0.25,0.44,0.63}{{#1}}}
+\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.25,0.44,0.63}{{#1}}}
+\newcommand{\StringTok}[1]{\textcolor[rgb]{0.25,0.44,0.63}{{#1}}}
+\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.25,0.44,0.63}{{#1}}}
+\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.73,0.40,0.53}{{#1}}}
+\newcommand{\ImportTok}[1]{{#1}}
+\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.38,0.63,0.69}{\textit{{#1}}}}
+\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.73,0.13,0.13}{\textit{{#1}}}}
+\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.38,0.63,0.69}{\textbf{\textit{{#1}}}}}
+\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.38,0.63,0.69}{\textbf{\textit{{#1}}}}}
+\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.00,0.44,0.13}{{#1}}}
+\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.02,0.16,0.49}{{#1}}}
+\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.10,0.09,0.49}{{#1}}}
+\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.00,0.44,0.13}{\textbf{{#1}}}}
+\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.40,0.40,0.40}{{#1}}}
+\newcommand{\BuiltInTok}[1]{{#1}}
+\newcommand{\ExtensionTok}[1]{{#1}}
+\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.74,0.48,0.00}{{#1}}}
+\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.49,0.56,0.16}{{#1}}}
+\newcommand{\RegionMarkerTok}[1]{{#1}}
+\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.38,0.63,0.69}{\textbf{\textit{{#1}}}}}
+\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.38,0.63,0.69}{\textbf{\textit{{#1}}}}}
+\newcommand{\AlertTok}[1]{\textcolor[rgb]{1.00,0.00,0.00}{\textbf{{#1}}}}
+\newcommand{\ErrorTok}[1]{\textcolor[rgb]{1.00,0.00,0.00}{\textbf{{#1}}}}
+\newcommand{\NormalTok}[1]{{#1}}
+\usepackage{graphicx,grffile}
+\makeatletter
+\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
+\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
+\makeatother
+% Scale images if necessary, so that they will not overflow the page
+% margins by default, and it is still possible to overwrite the defaults
+% using explicit options in \includegraphics[width, height, ...]{}
+\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
+\IfFileExists{parskip.sty}{%
+\usepackage{parskip}
+}{% else
+\setlength{\parindent}{0pt}
+\setlength{\parskip}{6pt plus 2pt minus 1pt}
+}
+
+% previously included by writedown in options.sty
+\setlength{\parindent}{1.25em}
+\setlength{\parskip}{.2em}
+\usepackage{etoolbox}
+\AtBeginEnvironment{quote}{\parskip 1em}
+
+\setlength{\emergencystretch}{3em}  % prevent overfull lines
+\providecommand{\tightlist}{%
+  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
+\setcounter{secnumdepth}{0}
+% Redefines (sub)paragraphs to behave more like sections
+\ifx\paragraph\undefined\else
+\let\oldparagraph\paragraph
+\renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}}
+\fi
+\ifx\subparagraph\undefined\else
+\let\oldsubparagraph\subparagraph
+\renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}}
+\fi
+% END OF CONFIG    ------------------------------------------
+
+% START OF CONTENT ------------------------------------------
+
+\title{$title$}
+$if(subtitle)$
+\providecommand{\subtitle}[1]{}
+\subtitle{$subtitle$}
+$endif$
+$if(author)$
+\author{$for(author)$$author$$sep$ \and $endfor$}
+$endif$
+$if(institute)$
+\providecommand{\institute}[1]{}
+\institute{$for(institute)$$institute$$sep$ \and $endfor$}
+$endif$
+\date{$date$}
+$if(logo)$
+\logo{\includegraphics{$logo$}}
+$endif$
+
+\begin{document}
+
+\maketitle
+
+\begin{abstract}
+$abstract$
+\end{abstract}
+
+\providecommand{\keywords}[1]{\textbf{\textit{Keywords---}} #1}
+$if(keywords)$
+    \keywords{$for(keywords)$$keywords$$sep$; $endfor$}
+$endif$
+
+\pagebreak[4]
+{
+\setcounter{tocdepth}{3}
+\tableofcontents
+}
+\pagebreak[4]
+
+$body$
+
+\end{document}