Tadej Janežhttps://tadej.ja.nez.si/2017-11-26T22:13:00+01:00Enabling updates-testing and epel-testing repositories when using Copr2017-11-26T22:13:00+01:002017-11-26T22:13:00+01:00Tadej Janežtag:tadej.ja.nez.si,2017-11-26:/enabling-testing-repos-in-copr.html
<p>When creating a Copr project, one can specify which chroots, i.e. which
distribution version and architecture combinations, to enable. Examples of
chroots are <code>fedora-27-x86_64</code>, <code>epel-7-ppc64le</code>, <code>fedora-rawhide-i386</code> etc.</p>
<p>However, there is no easy to use option to enable testing repositories, e.g.
the <a href="https://fedoraproject.org/wiki/Repositories#updates-testing"><code>updates-testing</code> repository</a> for Fedora
Branched and stable releases, when specifying the chroots.
Let me show you how you can easily achieve that by specifying an extra
repository with a parameterized URL.</p>
<p><a href="https://copr.fedorainfracloud.org">Copr</a> is a really useful <a href="https://getfedora.org/">Fedora</a> community build service which also hosts
<a href="http://dnf.readthedocs.io">dnf</a>/<a href="http://yum.baseurl.org/">yum</a> repositories of
the built packages.</p>
<p>When creating a Copr project, one can specify which chroots, i.e. which
distribution version and architecture combinations, to enable. Examples of
chroots are <code>fedora-27-x86_64</code>, <code>epel-7-ppc64le</code>, <code>fedora-rawhide-i386</code> etc.</p>
<p>However, there is no easy to use option to enable testing repositories, e.g.
the <a href="https://fedoraproject.org/wiki/Repositories#updates-testing"><code>updates-testing</code> repository</a> for Fedora
Branched and stable releases, when specifying the chroots.
Let me show you how you can easily achieve that by specifying an extra
repository with a parameterized URL.</p>
<p>When setting up a Copr project, one can also specify extra repositories to
enable (in the web UI, they are referred to as <em>External repositories</em>).</p>
<p>A nice feature of Copr is that the URLs of these extra repositories can be
parameterized with variables representing the name of the distribution
(<code>$distname</code>), release version (<code>$releasever</code>) and base architecture
(<code>$basearch</code>).</p>
<p>Using that, we can easily specify an extra repository that will correspond to
the appropriate testing repository of any given chroot.</p>
<p>Unfortunately, Fedora and <a href="https://fedoraproject.org/wiki/EPEL"><abbr title="Extra Packages for Enterprise Linux">EPEL</abbr></a> don't use
the same directory structure, so one can't use the same parameterized extra
repository URL.</p>
<p>If you want to enable the <code>updates-testing</code> repository in a Fedora chroot,
add the following extra repository URL to your Copr project:</p>
<div class="highlight"><pre><span></span>http://download.fedoraproject.org/pub/$distname/linux/updates/testing/$releasever/$basearch/
</pre></div>
<p>If you want to enable the <a href="https://fedoraproject.org/wiki/EPEL/testing"><code>epel-testing</code> repository</a> in an <abbr title="Extra Packages for Enterprise Linux">EPEL</abbr> chroot, add the
following extra repository URL to your Copr project:</p>
<div class="highlight"><pre><span></span>http://download.fedoraproject.org/pub/$distname/testing/$releasever/$basearch/
</pre></div>
<p>To do this using <a href="https://developer.fedoraproject.org/deployment/copr/copr-cli.html">Copr command line interface</a>, run the
following:</p>
<div class="highlight"><pre><span></span>copr-cli modify --repo <span class="s1">'<parameterized-extra-repository-URL>'</span>
</pre></div>
<p>For example, to enable the <code>updates-testing</code> repository in a Fedora chroot,
run:</p>
<div class="highlight"><pre><span></span>copr-cli modify --repo <span class="se">\</span>
<span class="s1">'http://download.fedoraproject.org/pub/$distname/linux/updates/testing/$releasever/$basearch/'</span>
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>You must put the parameterized extra repository URL in single quotes,
otherwise Bash will evaluate the variables (e.g. <code>$distname</code>) in the
current context, where they are not defined, and silently substitute them
with empty strings.</p>
</div>
<p>If your Copr project includes a Fedora and an <abbr title="Extra Packages for Enterprise Linux">EPEL</abbr> chroot, just add both,
the <code>updates-testing</code> and the <code>epel-testing</code> repository, as extra
repositories.</p>
<p>A build of the Copr project in a Fedora chroot will fail to fetch the
information about the (non-existent) <code>epel-testing</code> repository, but it will
successfully continue. The same goes for a build in an <abbr title="Extra Packages for Enterprise Linux">EPEL</abbr> chroot or a <a href="https://fedoraproject.org/wiki/Repositories#The_rawhide_repository">Fedora
rawhide</a>
chroot, which don't have a corresponding <code>updates-testing</code> repository.</p>Running Fedora 26 on Dell XPS 15 95602017-09-04T09:35:00+02:002017-09-04T09:35:00+02:00Tadej Janežtag:tadej.ja.nez.si,2017-09-04:/fedora-26-on-dell-xps-15-9560.html
<p>Recently, I got a new laptop, a <a href="http://www.dell.com/en-us/shop/dell-laptops/xps-15/spd/xps-15-9560-laptop">Dell XPS 15 9560</a> with
4k display, 32 GiBs of RAM and 1 TiB M.2 <abbr title="Solid-state drive">SSD</abbr> drive. Quite nice specs, aren't
they <img align="absmiddle" alt="😉" class="emojione" height="20px" src="https://cdn.jsdelivr.net/emojione/assets/svg/1f609.svg" title=":wink:" width="20px">?</p>
<p>Unfortunately, I wasn't able to purchase it without Windows pre-installed. But
they didn't last long since I wiped the hard drive and installed <a href="https://fedoramagazine.org/fedora-26-is-here/">Fedora 26</a> on it.</p>
<p>Let me briefly describe my experience and guide you through some trickier parts
of the setup.</p>
<p><em>Update (November 18, 2017): After receiving lots of feedback about issues with
setting up the graphics to an adequate level, I rewrote <a href="#setting-up-graphics">that part of the blog
post</a>. I've also added instructions on actually <a href="#enabling-the-use-of-normal-graphics-drivers">enabling
the use of <em>normal</em> graphics drivers
</a> since many people unknowingly
used Mesa's software rasterizer, which causes all sorts of issues.</em></p>
<p>Recently, I got a new laptop, a <a href="http://www.dell.com/en-us/shop/dell-laptops/xps-15/spd/xps-15-9560-laptop">Dell XPS 15 9560</a> with
4k display, 32 GiBs of RAM and 1 TiB M.2 <abbr title="Solid-state drive">SSD</abbr> drive. Quite nice specs, aren't
they <img align="absmiddle" alt="😉" class="emojione" height="20px" src="https://cdn.jsdelivr.net/emojione/assets/svg/1f609.svg" title=":wink:" width="20px">?</p>
<p>Unfortunately, I wasn't able to purchase it without Windows pre-installed. But
they didn't last long since I wiped the hard drive and installed <a href="https://fedoramagazine.org/fedora-26-is-here/">Fedora 26</a> on it.</p>
<p>Let me briefly describe my experience and guide you through some trickier parts
of the setup.</p>
<h2 id="configuring-uefi-to-work-with-fedora-26">Configuring <abbr title="Unified Extensible Firmware Interface">UEFI</abbr> to work with Fedora 26</h2>
<p>Before installing Fedora 26 on the system, it is necessary to modify some <abbr title="Unified Extensible Firmware Interface">UEFI</abbr>
settings. <a href="https://wiki.archlinux.org/index.php/Dell_XPS_15_9560#UEFI">They are nicely summed up in the ArchWiki page for this laptop</a>:</p>
<ul>
<li>Change the <em>SATA Mode</em> from the default <em>RAID</em> to <em>AHCI</em>. This will allow
Linux to detect the <abbr title="Non-volatile memory (NVM) Express">NVMe</abbr> <abbr title="Solid-state drive">SSD</abbr>.</li>
<li>Change <em>Fastboot</em> to <em>Thorough</em> in <em>POST Behaviour</em>. This will prevent
intermittent boot failures.</li>
</ul>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If you intend to run the proprietary NVIDIA graphics drivers, you must also
disable secure boot.</p>
</div>
<h2 id="booting-fedora-26-from-a-live-usb">Booting Fedora 26 from a live USB</h2>
<p>The naive approach to just boot the computer from a live USB with Fedora 26
Workstation image and wait for the live <a href="https://www.gnome.org/">GNOME</a> session
to appear didn't work. Fedora hangs after the <em>Started User Manager for UID
1000</em> <a href="https://www.freedesktop.org/software/systemd/man/systemd.unit.html">systemd unit</a> starts.</p>
<p>Fortunately, <a href="https://ask.fedoraproject.org/en/question/107784/f26-hanging-on-install-dell-xps-9560/?answer=107824#post-id-107824">Ask Fedora has an answer to this problem</a>,
which is to select <em>Troubleshooting</em> from <a href="https://www.gnu.org/software/grub/">GRUB</a>'s boot menu and then choose <em>Start in basic
graphics mode</em>. The live GNOME session should work fine after that.</p>
<h2 id="installing-fedora-26-to-the-hard-drive">Installing Fedora 26 to the hard drive</h2>
<p>This works smoothly. Just select <em>Install to Hard Drive</em> option to start
<a href="https://fedoraproject.org/wiki/Anaconda">Anaconda</a>, Fedora's installer, and
follow its instructions.</p>
<p>After installation, make sure to update all packages to get the latest
kernel (4.12 at the time of writing) and other important updates which will
make Fedora run smoother on this laptop.</p>
<h2 id="setting-up-graphics">Setting up graphics</h2>
<p>The laptop comes with a discrete <abbr title="Graphics processing unit">GPU</abbr>, the <a href="https://en.wikipedia.org/wiki/GeForce_10_series#GeForce_10_.2810xx.29_series_for_notebooks">NVIDIA GeForce GTX 1050
Mobile (GP107M)</a>,
and an integrated <abbr title="Graphics processing unit">GPU</abbr>, the <a href="https://en.wikipedia.org/wiki/Intel_HD_and_Iris_Graphics#Kaby_Lake">Intel HD Graphics 630 (Kaby Lake GT2)</a>.</p>
<h3 id="enabling-the-use-of-normal-graphics-drivers">Enabling the use of <em>normal</em> graphics drivers</h3>
<p>Since we needed to use the <em>basic graphics mode</em> when <a href="#booting-fedora-26-from-a-live-usb">booting Fedora 26 from
a live USB</a>, the installer "baked" this
information into the newly installed system. While this is a safe choice, i.e.
the system is guaranteed to successfully boot into graphics mode, it also means
that no advanced graphics features will work.
When I used the system in this mode, GNOME didn't detect that it runs on a
HiDPI screen and hence everything on the screen was too small. Connecting an
external monitor also didn't work and moving around the UI elements was slow
and sluggish.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If you want to check if you are running in the <em>basic graphics mode</em>, go to
GNOME's Settings and open the <em>Details</em> dialog. If you see <code>Gallium 0.4 on
llvmpipe (...)</code> or similar under <em>Graphics</em>, this means the system is
running in the <em>basic graphics mode</em>.</p>
</div>
<p>The reason for this is that the installer added the <code>nomodeset</code> kernel command
line option to GRUB's settings. Passing the <code>nomodeset</code> option to the kernel
disables <a href="https://fedoraproject.org/wiki/Features/KernelModesetting">Kernel Mode setting (KMS)</a> which in turn forces
the system to fallback to using <a href="https://www.mesa3d.org/llvmpipe.html">Mesa's Gallium llvmpipe driver</a> which is a software rasterizer that runs
on the <abbr title="Central processing unit">CPU</abbr>.</p>
<p>To enable <em>normal</em> graphics drivers, edit <code>/etc/default/grub</code> and remove
<code>nomodeset</code> from the contents of the <code>GRUB_CMDLINE_LINUX</code> variable.</p>
<p>Afterwards, run <code>sudo grub2-mkconfig -o /etc/grub2-efi.cfg</code> to regenerate
GRUB's configuration and reboot the system.</p>
<h3 id="a-rocky-road-when-trying-to-use-nvidia-discrete-gpu-with-nouveau-drivers">A rocky road when trying to use NVIDIA discrete <abbr title="Graphics processing unit">GPU</abbr> with Nouveau drivers</h3>
<p>Unfortunately, NVIDIA GeForce GTX 1050 Mobile (GP107M) doesn't work well with
the <a href="https://nouveau.freedesktop.org/wiki/">Nouveau drivers</a> yet (as of kernel
4.12).</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Initial support for NVIDIA's GP107 devices <a href="https://git.kernel.org/linus/b2c4ef70790cee37f243af2b303727394edae1d5">has</a>
<a href="https://git.kernel.org/linus/2ebd42bc28525da52162425ecd7472846b78584d">been</a>
<a href="https://git.kernel.org/linus/5429f82f341524deb9f66193892a69dea2f862a3">merged</a> into
kernel 4.12.</p>
</div>
<p>Practically, this means that the system will fail to boot (similarly as
<a href="#booting-fedora-26-from-a-live-usb">described above</a>) if one doesn't disable
Nouveau's power management by passing the <code>nouveau.runpm=0</code> parameter to the
kernel at boot. More information in <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1447677">Red Hat's Bugzilla #1447677</a>.</p>
<p>Unfortunately, that's not enough. After successfully booting the system into
a graphical session, I soon started encountering soft kernel lockups due to
Nouveau driver. The logs showed the following:</p>
<div class="highlight"><pre><span></span>NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [plymouthd:423]
[ ... output trimmed ... ]
CPU: 1 PID: 423 Comm: plymouthd Not tainted 4.12.9-300.fc26.x86_64 #1
Hardware name: Dell Inc. XPS 15 9560/05FFDN, BIOS 1.3.4 06/08/2017
task: ffff92a115bea640 task.stack: ffffa79943d7c000
[ ... output trimmed ... ]
Call Trace:
? nv04_timer_read+0x48/0x60 [nouveau]
nvkm_timer_read+0xf/0x20 [nouveau]
nvkm_pmu_reset+0x71/0x170 [nouveau]
nvkm_pmu_preinit+0x12/0x20 [nouveau]
nvkm_subdev_preinit+0x34/0x110 [nouveau]
nvkm_device_init+0x60/0x270 [nouveau]
nvkm_udevice_init+0x48/0x60 [nouveau]
nvkm_object_init+0x3f/0x190 [nouveau]
nvkm_object_init+0xa3/0x190 [nouveau]
nvkm_client_resume+0xe/0x10 [nouveau]
nvif_client_resume+0x17/0x20 [nouveau]
nouveau_do_resume+0x40/0xe0 [nouveau]
nouveau_pmops_runtime_resume+0x91/0x150 [nouveau]
[ ... output trimmed ... ]
</pre></div>
<p>Looking through <a href="https://fedoramagazine.org/systemd-using-journal/">systemd journal</a> soon revealed other kernel
tracebacks related to the Nouveau driver:</p>
<div class="highlight"><pre><span></span><span class="n">nouveau</span> <span class="mo">0000</span><span class="o">:</span><span class="mo">01</span><span class="o">:</span><span class="mf">00.0</span><span class="o">:</span> <span class="nl">bus</span><span class="p">:</span> <span class="n">MMIO</span> <span class="n">read</span> <span class="n">of</span> <span class="mo">00000000</span> <span class="n">FAULT</span> <span class="n">at</span> <span class="mi">409800</span> <span class="p">[</span> <span class="n">TIMEOUT</span> <span class="p">]</span>
<span class="p">[</span><span class="nl">drm</span><span class="p">:</span><span class="n">i915_gem_idle_work_handler</span> <span class="p">[</span><span class="n">i915</span><span class="p">]]</span> <span class="o">*</span><span class="n">ERROR</span><span class="o">*</span> <span class="n">Timeout</span> <span class="n">waiting</span> <span class="k">for</span> <span class="n">engines</span> <span class="n">to</span> <span class="n">idle</span>
<span class="n">nouveau</span> <span class="mo">0000</span><span class="o">:</span><span class="mo">01</span><span class="o">:</span><span class="mf">00.0</span><span class="o">:</span> <span class="n">timeout</span>
<span class="o">------------</span><span class="p">[</span> <span class="n">cut</span> <span class="n">here</span> <span class="p">]</span><span class="o">------------</span>
<span class="nl">WARNING</span><span class="p">:</span> <span class="nl">CPU</span><span class="p">:</span> <span class="mi">1</span> <span class="nl">PID</span><span class="p">:</span> <span class="mi">4100</span> <span class="n">at</span> <span class="n">drivers</span><span class="o">/</span><span class="n">gpu</span><span class="o">/</span><span class="n">drm</span><span class="o">/</span><span class="n">nouveau</span><span class="o">/</span><span class="n">nvkm</span><span class="o">/</span><span class="n">engine</span><span class="o">/</span><span class="n">gr</span><span class="o">/</span><span class="n">gf100</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">1501</span> <span class="n">gf100_gr_init_ctxctl</span><span class="o">+</span><span class="mh">0x81f</span><span class="o">/</span><span class="mh">0x9a0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="p">[</span> <span class="p">...</span> <span class="n">output</span> <span class="n">trimmed</span> <span class="p">...</span> <span class="p">]</span>
<span class="n">Hardware</span> <span class="nl">name</span><span class="p">:</span> <span class="n">Dell</span> <span class="n">Inc</span><span class="p">.</span> <span class="n">XPS</span> <span class="mi">15</span> <span class="mi">9560</span><span class="o">/</span><span class="mf">05FF</span><span class="n">DN</span><span class="p">,</span> <span class="n">BIOS</span> <span class="mf">1.3.4</span> <span class="mo">06</span><span class="o">/</span><span class="mi">08</span><span class="o">/</span><span class="mi">2017</span>
<span class="nl">task</span><span class="p">:</span> <span class="n">ffff8d8261e00000</span> <span class="n">task</span><span class="p">.</span><span class="nl">stack</span><span class="p">:</span> <span class="n">ffffa5c91190c000</span>
<span class="p">[</span> <span class="p">...</span> <span class="n">output</span> <span class="n">trimmed</span> <span class="p">...</span> <span class="p">]</span>
<span class="n">Call</span> <span class="nl">Trace</span><span class="p">:</span>
<span class="n">gp100_gr_init</span><span class="o">+</span><span class="mh">0x6f0</span><span class="o">/</span><span class="mh">0x720</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">gf100_gr_init_</span><span class="o">+</span><span class="mh">0x55</span><span class="o">/</span><span class="mh">0x60</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_gr_init</span><span class="o">+</span><span class="mh">0x17</span><span class="o">/</span><span class="mh">0x20</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_engine_init</span><span class="o">+</span><span class="mh">0x68</span><span class="o">/</span><span class="mh">0x1f0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_subdev_init</span><span class="o">+</span><span class="mh">0xb0</span><span class="o">/</span><span class="mh">0x200</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_engine_ref</span><span class="o">+</span><span class="mh">0x4f</span><span class="o">/</span><span class="mh">0x70</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_ioctl_new</span><span class="o">+</span><span class="mh">0x2b4</span><span class="o">/</span><span class="mh">0x300</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="o">?</span> <span class="n">nvkm_fifo_chan_dtor</span><span class="o">+</span><span class="mh">0xe0</span><span class="o">/</span><span class="mh">0xe0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="o">?</span> <span class="n">gf100_gr_init_fw</span><span class="p">.</span><span class="n">isra</span><span class="mf">.8</span><span class="o">+</span><span class="mh">0x50</span><span class="o">/</span><span class="mh">0x50</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_ioctl</span><span class="o">+</span><span class="mh">0x118</span><span class="o">/</span><span class="mh">0x280</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_client_ioctl</span><span class="o">+</span><span class="mh">0x12</span><span class="o">/</span><span class="mh">0x20</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvif_client_ioctl</span><span class="o">+</span><span class="mh">0x26</span><span class="o">/</span><span class="mh">0x30</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">usif_ioctl</span><span class="o">+</span><span class="mh">0x637</span><span class="o">/</span><span class="mh">0x750</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nouveau_drm_ioctl</span><span class="o">+</span><span class="mh">0xaf</span><span class="o">/</span><span class="mh">0xc0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="p">[</span> <span class="p">...</span> <span class="n">output</span> <span class="n">trimmed</span> <span class="p">...</span> <span class="p">]</span>
<span class="o">---</span><span class="p">[</span> <span class="n">end</span> <span class="n">trace</span> <span class="mi">1</span><span class="n">ee905135b51d0af</span> <span class="p">]</span><span class="o">---</span>
<span class="n">nouveau</span> <span class="mo">0000</span><span class="o">:</span><span class="mo">01</span><span class="o">:</span><span class="mf">00.0</span><span class="o">:</span> <span class="nl">gr</span><span class="p">:</span> <span class="n">init</span> <span class="n">failed</span><span class="p">,</span> <span class="o">-</span><span class="mi">16</span>
<span class="n">nouveau</span> <span class="mo">0000</span><span class="o">:</span><span class="mo">01</span><span class="o">:</span><span class="mf">00.0</span><span class="o">:</span> <span class="n">timeout</span>
<span class="o">------------</span><span class="p">[</span> <span class="n">cut</span> <span class="n">here</span> <span class="p">]</span><span class="o">------------</span>
<span class="nl">WARNING</span><span class="p">:</span> <span class="nl">CPU</span><span class="p">:</span> <span class="mi">2</span> <span class="nl">PID</span><span class="p">:</span> <span class="mi">4100</span> <span class="n">at</span> <span class="n">drivers</span><span class="o">/</span><span class="n">gpu</span><span class="o">/</span><span class="n">drm</span><span class="o">/</span><span class="n">nouveau</span><span class="o">/</span><span class="n">nvkm</span><span class="o">/</span><span class="n">subdev</span><span class="o">/</span><span class="n">mmu</span><span class="o">/</span><span class="n">gf100</span><span class="p">.</span><span class="nl">c</span><span class="p">:</span><span class="mi">190</span> <span class="n">gf100_vm_flush</span><span class="o">+</span><span class="mh">0x1ab</span><span class="o">/</span><span class="mh">0x1c0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="p">[</span> <span class="p">...</span> <span class="n">output</span> <span class="n">trimmed</span> <span class="p">...</span> <span class="p">]</span>
<span class="n">Hardware</span> <span class="nl">name</span><span class="p">:</span> <span class="n">Dell</span> <span class="n">Inc</span><span class="p">.</span> <span class="n">XPS</span> <span class="mi">15</span> <span class="mi">9560</span><span class="o">/</span><span class="mf">05FF</span><span class="n">DN</span><span class="p">,</span> <span class="n">BIOS</span> <span class="mf">1.3.4</span> <span class="mo">06</span><span class="o">/</span><span class="mi">08</span><span class="o">/</span><span class="mi">2017</span>
<span class="nl">task</span><span class="p">:</span> <span class="n">ffff8d8261e00000</span> <span class="n">task</span><span class="p">.</span><span class="nl">stack</span><span class="p">:</span> <span class="n">ffffa5c91190c000</span>
<span class="p">[</span> <span class="p">...</span> <span class="n">output</span> <span class="n">trimmed</span> <span class="p">...</span> <span class="p">]</span>
<span class="n">Call</span> <span class="nl">Trace</span><span class="p">:</span>
<span class="n">nvkm_vm_unmap_at</span><span class="o">+</span><span class="mh">0xbc</span><span class="o">/</span><span class="mh">0x100</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nvkm_vm_unmap</span><span class="o">+</span><span class="mh">0x1b</span><span class="o">/</span><span class="mh">0x20</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">nouveau_gem_object_close</span><span class="o">+</span><span class="mh">0x1aa</span><span class="o">/</span><span class="mh">0x1c0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="n">drm_gem_object_release_handle</span><span class="o">+</span><span class="mh">0x4b</span><span class="o">/</span><span class="mh">0x90</span> <span class="p">[</span><span class="n">drm</span><span class="p">]</span>
<span class="n">drm_gem_handle_delete</span><span class="o">+</span><span class="mh">0x58</span><span class="o">/</span><span class="mh">0x80</span> <span class="p">[</span><span class="n">drm</span><span class="p">]</span>
<span class="n">drm_gem_close_ioctl</span><span class="o">+</span><span class="mh">0x20</span><span class="o">/</span><span class="mh">0x30</span> <span class="p">[</span><span class="n">drm</span><span class="p">]</span>
<span class="n">drm_ioctl</span><span class="o">+</span><span class="mh">0x213</span><span class="o">/</span><span class="mh">0x4d0</span> <span class="p">[</span><span class="n">drm</span><span class="p">]</span>
<span class="o">?</span> <span class="n">drm_gem_handle_create</span><span class="o">+</span><span class="mh">0x40</span><span class="o">/</span><span class="mh">0x40</span> <span class="p">[</span><span class="n">drm</span><span class="p">]</span>
<span class="o">?</span> <span class="n">sock_write_iter</span><span class="o">+</span><span class="mh">0x8c</span><span class="o">/</span><span class="mh">0xf0</span>
<span class="n">nouveau_drm_ioctl</span><span class="o">+</span><span class="mh">0x72</span><span class="o">/</span><span class="mh">0xc0</span> <span class="p">[</span><span class="n">nouveau</span><span class="p">]</span>
<span class="o">---</span><span class="p">[</span> <span class="n">end</span> <span class="n">trace</span> <span class="mi">1</span><span class="n">ee905135b51d0b0</span> <span class="p">]</span><span class="o">---</span>
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>I've also dipped my toes into setting up a hybrid graphics setup that would
use <a href="http://www.nvidia.com/download/driverResults.aspx/126185/en-us">Nvidia's proprietary driver</a> as
<a href="https://negativo17.org/nvidia-driver/">packaged by negativo17</a> in
combination with the open source driver for Intel's integrated <abbr title="Graphics processing unit">GPU</abbr>.</p>
<p>I haven't been successful in the limited time I devoted this, but if you
want to try this out I suggest you read <a href="https://blogs.gnome.org/uraeus/">Christian Schaller's (manager of
Red Hat's Desktop team)</a> blog post on
<a href="https://blogs.gnome.org/uraeus/2017/07/13/fedora-workstation-26-is-out/">Fedora Workstation 26</a>.
The post describes the advancements the Red Hat Desktop team has made in
making the hybrid graphics setups easy to use, even if one uses the
proprietary Nvidia driver.</p>
</div>
<h3 id="flying-high-after-starting-to-use-intels-integrated-gpu">Flying high after starting to use Intel's integrated <abbr title="Graphics processing unit">GPU</abbr></h3>
<p>After encountering a Nouveau developer's (Rhys Kidd) <a href="https://bugs.freedesktop.org/show_bug.cgi?id=102275#c1">answer to a bug report
for a similar issue that I described above on freedesktop.org's Bugzilla</a> on Aug 17, 2017:</p>
<blockquote>
<p>Nouveau has difficulties with the GTX 1050 Mobile (GP107/NV137) around power
state up/down. I'd suggest you continue to follow <a href="https://bugs.freedesktop.org/show_bug.cgi?id=100228">that bug report</a> for further updates.</p>
<p>Whilst <code>nouveau.runpm=0</code> assists a little bit, for now, to improve the
experience on the XPS 9560 you should use <code>nouveau.modeset=0</code> (although that
does mean the Nvidia <abbr title="Graphics processing unit">GPU</abbr> is unavailable for use). This suggestion might
change with future kernel releases.</p>
</blockquote>
<p>I decided to follow his suggestion and disable the Nouveau driver by adding
<code>nouveau.modeset=0</code> to the kernel's command line options. In my observations so
far, everything is working smootly:</p>
<ul>
<li>automatic scaling of GNOME's interface (it even works if you use laptop's
built-in HiDPI display in combination with an external non-HiDPI display),</li>
<li>fast day to day work (i.e. no sluggishness),</li>
<li>suspend and resume (works even after many suspend and resume cycles or if
you resume the laptop with/without an external display),</li>
<li>battery life is great (I haven't done any proper benchmarks, but it easily
lasts around 4 hours of my ordinary programming work).</li>
</ul>
<p>To make this permament, edit <code>/etc/default/grub</code> and append <code>nouveau.modeset=0</code>
to the contents of the <code>GRUB_CMDLINE_LINUX</code> variable.</p>
<p>Afterwards, run <code>sudo grub2-mkconfig -o /etc/grub2-efi.cfg</code> to regenerate
GRUB's configuration.</p>
<h2 id="updating-firmware-to-version-134">Updating firmware to version 1.3.4+</h2>
<p>It is important to upgrade the laptop's firmware to (at least) <a href="https://www.notebookcheck.net/XPS-15-9560-gets-BIOS-update-fixing-Skylake-Kaby-Lake-microcode-bug.232054.0.html">version 1.3.4</a>
since it fixes <a href="https://lwn.net/Articles/726496/">a serious issue with Intel's Skylake and Kaby Lake
architectures where the <abbr title="Central processing unit">CPU</abbr> could dangerously misbehave when hyper-threading is
enabled</a>.</p>
<p>Fortunately, Dell has put Linux users in the prime seat and offers seamless
firmware updates on Linux via <a href="https://blogs.gnome.org/hughsie/">Richard Hughes</a>'s excellent <a href="http://www.fwupd.org/">fwupd tool</a>.</p>
<p>To update the firmware using command line, simply run:</p>
<div class="highlight"><pre><span></span>fwupdmgr refresh
fwupdmgr get-updates
</pre></div>
<p>to see the list of available updates. To download the available update and
install it after reboot, run:</p>
<div class="highlight"><pre><span></span>fwupdmgr update
</pre></div>
<p>Alternatively, you can also update the firmware using GNOME's <a href="https://wiki.gnome.org/Apps/Software">Software app</a>.</p>
<h1 id="issues-with-wireless-performance-on-kernel-412">Issues with wireless performance on kernel 4.12</h1>
<p>I've experienced <a href="https://bodhi.fedoraproject.org/updates/kernel-4.12.5-300.fc26#comment-645648">an order of magnitude wireless performance drop after
upgrading from <code>kernel-4.11.11-300.fc26.x86_64</code> to <code>kernel-4.12.5-300.fc26</code></a>
with the Qualcomm Atheros QCA6174 802.11ac wireless network adapter that is
installed in the laptop.
The reported wireless link speed is in 1 - 6 Mbps range, which is very low.</p>
<p>I haven't had time to investigate this further yet.</p>
<h1 id="conclusion">Conclusion</h1>
<p>Overall, my experience (minus the problems with NVIDIA's graphics card and
its open-source drivers and the wireless performance issue) has been quite
positive.</p>
<p>Sadly, getting all the features of a new laptop to work seamlessly after it
launches is <em>not there yet</em> and one still has to be quite knowledgable to make
the most out of the system.</p>
<p>However, I'm optimistic and I think it's getting better. For example, I was
pleasantly surprised how GNOME seamlessly re-scales the interface after
connecting/disconnecting a non-HiDPI monitor. Suspend also works without a
hitch!</p>Fedora EuroPython 2017 Attendee badge2017-08-27T15:51:00+02:002017-08-27T15:51:00+02:00Tadej Janežtag:tadej.ja.nez.si,2017-08-27:/fedora-europython2017-attendee-badge.html
<p>A little late, but no less cool, <a href="https://badges.fedoraproject.org/badge/europython-2017-attendee">Fedora EuroPython 2017 Attendee badge</a> is available
<img align="absmiddle" alt="🎉" class="emojione" height="20px" src="https://cdn.jsdelivr.net/emojione/assets/svg/1f389.svg" title=":tada:" width="20px">!</p>
<p><img alt="Fedora EuroPython 2017 Attendee badge" src="https://tadej.ja.nez.si/images/fedora-europython2017-attendee-badge.svg"></p>
<p>A little late, but no less cool, <a href="https://badges.fedoraproject.org/badge/europython-2017-attendee">Fedora EuroPython 2017 Attendee badge</a> is available
<img align="absmiddle" alt="🎉" class="emojione" height="20px" src="https://cdn.jsdelivr.net/emojione/assets/svg/1f389.svg" title=":tada:" width="20px">!</p>
<p><img alt="Fedora EuroPython 2017 Attendee badge" src="https://tadej.ja.nez.si/images/fedora-europython2017-attendee-badge.svg"></p>
<p>I've just been given the rights to award the badge, so if you attended <a href="https://ep2017.europython.eu/en/">this
years EuroPython in Rimini</a> and would like to
have this badge, please contact me and I'll be happy to award it to you.</p>
<p>When you write to me, please don't forget to include:</p>
<ul>
<li>your <a href="https://admin.fedoraproject.org/accounts/"><abbr title="Fedora Accounts System">FAS</abbr></a> username,</li>
<li>a proof that you attended the conference (e.g. a photo from the conference,
a photo of your conference badge, a link to your talk, ...).</li>
</ul>
<p>For example, here is a photo of me and my pimped up conference badge <img align="absmiddle" alt="😃" class="emojione" height="20px" src="https://cdn.jsdelivr.net/emojione/assets/svg/1f603.svg" title=":smiley:" width="20px">:</p>
<p><img alt="Me holding my EuroPython 2017 badge" src="https://tadej.ja.nez.si/images/europyhon2017-photo.jpg"></p>
<p>And please share this information with other EuroPython 2017 attendees!</p>Backing up Android's /data/media (i.e. internal storage) using adb and TWRP2017-07-30T12:28:00+02:002017-07-30T12:28:00+02:00Tadej Janežtag:tadej.ja.nez.si,2017-07-30:/android-internal-storage-backup.html
<p>Recently I had to clone my <a href="https://en.wikipedia.org/wiki/Nexus_4">Nexus 4</a> to
another Nexus 4 device since my Nexus 4's front glass broke :-(.</p>
<p>At first, the task seemed quite straight-forward. Install the latest version of
<a href="https://twrp.me/"><abbr title="Team Win Recovery Project">TWRP</abbr></a>'s recovery on the phone, backup all partitions using
<abbr title="Team Win Recovery Project">TWRP</abbr>, transfer them to the other device and restore them.</p>
<p>But thing are never as easy as them seem, are they?</p>
<p>Recently I had to clone my <a href="https://en.wikipedia.org/wiki/Nexus_4">Nexus 4</a> to
another Nexus 4 device since my Nexus 4's front glass broke :-(.</p>
<p>At first, the task seemed quite straight-forward. Install the latest version of
<a href="https://twrp.me/"><abbr title="Team Win Recovery Project">TWRP</abbr></a>'s recovery on the phone, backup all partitions using
<abbr title="Team Win Recovery Project">TWRP</abbr>, transfer them to the other device and restore them.</p>
<p>But thing are never as easy as them seem, are they?</p>
<h2 id="problems-with-the-straight-forward-approach">Problems with the straight-forward approach</h2>
<p>It turns out that <strong>creating</strong> a <strong>backup of</strong> the <strong>user data partition</strong>
using <abbr title="Team Win Recovery Project">TWRP</abbr> <strong>will NOT include <code>/data/media</code></strong> (i.e. your <strong>internal storage</strong>).
That means if you save photos or data on the internal storage (some apps will
save data there as well), they will be not included in a <abbr title="Team Win Recovery Project">TWRP</abbr> backup.</p>
<p>This is <a href="https://twrp.me/faq/backupexclusions.html">explained in <abbr title="Team Win Recovery Project">TWRP</abbr>'s FAQ</a>
and <abbr title="Team Win Recovery Project">TWRP</abbr>'s owner also <a href="https://github.com/TeamWin/Team-Win-Recovery-Project/issues/276#issuecomment-239172861">explained why they exclude <code>/data/media</code> from the backup
</a>
in a <a href="https://github.com/TeamWin/Team-Win-Recovery-Project/issues/276">heatedly debated GitHub issue on (not) excluding <code>/data/media</code> from
<abbr title="Team Win Recovery Project">TWRP</abbr>'s backups</a>.</p>
<p>There is even an <a href="https://github.com/TeamWin/Team-Win-Recovery-Project/issues/176">old open issue on GitHub</a> that asks for
an implementation of an option to backup <code>/data/media</code> using <abbr title="Team Win Recovery Project">TWRP</abbr>, but there has
been no updates since Apr 15, 2015 (as of writing this blog post).</p>
<h2 id="simple-command-line-solution-using-adb-and-tar">Simple command-line solution using <code>adb</code> and <code>tar</code></h2>
<p>Browsing the internet for a solution, I found a <a href="https://forum.xda-developers.com/android/software-hacking/how-to-backup-compressed-data-android-t3464777">nice thread on XDA-developers</a>
explaining how to backup data from an Android device booted into <abbr title="Team Win Recovery Project">TWRP</abbr> to a
computer.</p>
<p>The <em>trick</em> is to use <a href="https://android.googlesource.com/platform/system/core/+/5d9d434efadf1c535c7fea634d5306e18c68ef1f"><code>adb exec-out</code> command</a>
, an undocumented <a href="https://developer.android.com/studio/command-line/adb.html"><code>adb</code> command</a> that works like
<code>adb shell</code> but without creating a pseudoterminal (<code>pty</code>), which would
otherwise mangle binary data.</p>
<p>To backup <code>/data/media</code> (i.e. internal storage), first reboot your phone into
<abbr title="Team Win Recovery Project">TWRP</abbr> recovery.</p>
<p>Then execute the following:</p>
<div class="highlight"><pre><span></span>adb exec-out <span class="s1">'tar --create --exclude=data/media/0/TWRP \</span>
<span class="s1"> data/media/0 2>/backup-errors.txt | \</span>
<span class="s1"> gzip'</span> <span class="p">|</span> <span class="se">\</span>
dd <span class="nv">of</span><span class="o">=</span><backup-name>-<span class="k">$(</span>date +%Y%m%d<span class="k">)</span>.tar.gz <span class="o">&&</span> <span class="se">\</span>
adb shell cat /backup-errors.txt
</pre></div>
<p>replacing <code><backup-name></code> with your desired backup name, e.g.
<code>tadej-nexus4-sdcard_backup</code>.</p>
<h2 id="which-compression-algorithm-to-use">Which compression algorithm to use?</h2>
<p>A natural question that comes to mind is, which compression algorithm would
perform best in an overall sense, i.e. with respect to compression ratio and
the time it takes to create and transfer the archive.</p>
<p>Since we are just piping the (compressed) tar archive from the phone to the
computer, we can easily swap between creating the compressed archive on the
phone or on the computer. Compressing the archive on the computer would
not help reducing backup time if the bandwidth is the limit, but it may help
reducing the backup size.
On the other hand, if compression on the phone is CPU-bound, then performing
the compression on the computer will help reduce the backup time.</p>
<p>The variant of the command that compresses files on the computer is:</p>
<div class="highlight"><pre><span></span>adb exec-out <span class="s1">'tar --create --exclude=data/media/0/TWRP \</span>
<span class="s1"> data/media/0 2>/backup-errors.txt'</span> <span class="p">|</span> <span class="se">\</span>
<compression-command> <span class="p">|</span> <span class="se">\</span>
dd <span class="nv">of</span><span class="o">=</span><backup-name>-<span class="k">$(</span>date +%Y%m%d<span class="k">)</span>.tar.<compression-extension> <span class="o">&&</span> <span class="se">\</span>
adb shell cat /backup-errors.txt
</pre></div>
<p>where <code><compression-command></code> is one of <code>gzip</code>, <code>bzip2</code> and <code>xz</code> (optionally,
with extra arguments) and <code><compression-extension></code> is the chosen compression
tool's extension, i.e. <code>gz</code>, <code>bz2</code> or <code>xz</code>.</p>
<p>The results of backing up my phone's internal storage with different
compression commands and different execution places are the following (<a href="#what-kind-of-data-was-being-compressed">read
more details about the data being backed up in the next section
</a>):</p>
<table>
<thead>
<tr>
<th align="left">Compression command</th>
<th align="center">Compression execution</th>
<th align="right">Backup size</th>
<th align="right">Size compared to best</th>
<th align="right">Backup time</th>
<th align="right">Time compared to best</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><em>without</em></td>
<td align="center"><em>N/A</em></td>
<td align="right">3558 MiB</td>
<td align="right">+4.8%</td>
<td align="right">878 s</td>
<td align="right">+61%</td>
</tr>
<tr>
<td align="left"><code>gzip</code></td>
<td align="center">on the phone</td>
<td align="right">3428 MiB</td>
<td align="right">+1.0%</td>
<td align="right">549 s</td>
<td align="right">+1%</td>
</tr>
<tr>
<td align="left"><code>gzip --best</code></td>
<td align="center">on the phone</td>
<td align="right">3428 MiB</td>
<td align="right">+1.0%</td>
<td align="right"><strong>545 s</strong></td>
<td align="right"><em>N/A</em></td>
</tr>
<tr>
<td align="left"><code>bzip2</code></td>
<td align="center">on the phone</td>
<td align="right">3411 MiB</td>
<td align="right">+0.5%</td>
<td align="right">4236 s</td>
<td align="right">+677%</td>
</tr>
<tr>
<td align="left"><code>gzip</code></td>
<td align="center">on the computer</td>
<td align="right">3425 MiB</td>
<td align="right">+0.9%</td>
<td align="right">982 s</td>
<td align="right">+80%</td>
</tr>
<tr>
<td align="left"><code>bzip2</code></td>
<td align="center">on the computer</td>
<td align="right">3411 MiB</td>
<td align="right">+0.5%</td>
<td align="right">824 s</td>
<td align="right">+51%</td>
</tr>
<tr>
<td align="left"><code>xz</code></td>
<td align="center">on the computer</td>
<td align="right">3402 MiB</td>
<td align="right">+0.2%</td>
<td align="right">1566 s</td>
<td align="right">+187%</td>
</tr>
<tr>
<td align="left"><code>xz --threads=0</code></td>
<td align="center">on the computer</td>
<td align="right">3404 MiB</td>
<td align="right">+0.3%</td>
<td align="right">811 s</td>
<td align="right">+48%</td>
</tr>
<tr>
<td align="left"><code>xz --best --threads=0</code></td>
<td align="center">on the computer</td>
<td align="right"><strong>3395 MiB</strong></td>
<td align="right"><em>N/A</em></td>
<td align="right">895 s</td>
<td align="right">+64%</td>
</tr>
</tbody>
</table>
<p>The <strong>overall winner</strong> is clearly <strong>using <code>gzip</code></strong> and <strong>performing compression
on the phone</strong>. The time needed to perform the backup is the smallest by a
large margin while the backup size is only 1% larger compared to the best
compression achieved with <code>xz --best</code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Benchmarks were conducted on my Nexus 4 running <a href="https://eu.dl.twrp.me/mako/"><abbr title="Team Win Recovery Project">TWRP</abbr> 3.1.1-0
(twrp-3.1.1-0-mako.img)</a> connected to my <a href="https://en.wikipedia.org/wiki/HP_ZBook">HP
ZBook 15 G2 workstation</a> running
<a href="https://getfedora.org/en/workstation/">Fedora 25 Workstation</a>.</p>
<p><a href="https://twrp.me/site/update/2017/05/19/twrp-3.1.1-0-released.html"><abbr title="Team Win Recovery Project">TWRP</abbr> 3.1.1-0</a> is based
on <a href="https://omnirom.org/">OmniROM, a community developed Android derivative</a>, 7.1, which in turn is based on <a href="https://source.android.com/"><abbr title="Android Open Source Project">AOSP</abbr></a> 7.1.2.</p>
<p><abbr title="Team Win Recovery Project">TWRP</abbr> <a href="https://github.com/omnirom/android_bootable_recovery/blob/56cf564/Android.mk#L508-L510">doesn't use the <code>gzip</code> command provided by BusyBox</a>.
Rather, it uses the one provided by <a href="https://github.com/omnirom/android_bootable_recovery/tree/b523650/pigz"><code>pigz</code>, a parallel implementation of
gzip</a>.</p>
<p><abbr title="Team Win Recovery Project">TWRP</abbr>'s <a href="https://github.com/omnirom/android_external_busybox/blob/android-7.1/busybox-full.links#L15"><code>bzip2</code> and <code>xz</code> commands are provided by Busybox</a>.
Note that <a href="https://github.com/omnirom/android_external_busybox/tree/android-7.1">OmniROM still uses Busybox</a>,
while <a href="https://lwn.net/Articles/629362/"><abbr title="Android Open Source Project">AOSP</abbr> switched to toybox at the end of 2014</a>.</p>
<p>The <code>xz</code> command provided by this version of BusyBox, v1.22.1 bionic,
doesn't support compression. Note that one can pass the <code>-J</code> argument when
creating an archive with the <code>tar</code> command but it will <em>silently</em> create an
uncompressed tar archive.</p>
<p>Running <code>xz --best --threads=0</code> <strong>requires lots of RAM</strong>. More precisely,
it requires <strong>674 MiB per thread</strong>. For example, on my quad-core CPU with
hyperthreading enabled, it requires 5.3 GiB of RAM.</p>
</div>
<h2 id="what-kind-of-data-was-being-compressed">What kind of data was being compressed?</h2>
<p>To be fair and transparent when comparing different compression algorithms, it
is necessary to also tell something about the data being compressed.</p>
<p>The data being compressed is what I accumulated in my phone's user data
partition (i.e. internal storage) over the course of 4 years. Of course, I also
deleted data regularly, otherwise I would run of free space.</p>
<p>This is what makes up the 3558 MiBs of my phone's internal storage:</p>
<p><img alt="My phone's internal storage usage by file type" src="https://tadej.ja.nez.si/images/android-internal-storage-backup-usage.svg"></p>
<p>The largest portion is used by JPEG images (mostly pictures taken with the
phone), followed by MP4 videos (also mostly videos taken with the phone),
followed by MPEG and Ogg audio files (my music collection). The rest only
occupies 10 % of the phone's internal storage.
In my opinion, this can serve as a good real-life benchmark of backing up a
phone's internal stoarge.</p>
<p>This also explains why there were so <a href="#which-compression-algorithm-to-use">little differences between different
compression algorithms in the resulting backup size
</a> (tar archive without compression was
only 4.8 % larger that the smallest tar archive compressed with <code>xz --best</code>).
JPEG, MP4, MPEG audio, i.e. MP3, and Ogg Vorbis are all file formats that use
special-purpose compression techniques to reduce the size of files so they
cannot be further compressed to any significant extent.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>To analyse my phone's internal storage usage by file type I used
<a href="https://github.com/lemonsqueeze/disk_usage_by_file_type">lemonsqueeze's <code>disk_usage_by_file_type</code> Bash script</a>.</p>
<p>To draw the pie chart, I used <a href="https://matplotlib.org/">matplotlib</a>
(<a href="https://tadej.ja.nez.si/scripts/android-internal-storage-backup-usage.py">source</a>).</p>
</div>Don't mix yum/dnf and pip for installation of system-wide Python packages2017-01-13T11:43:00+01:002017-01-13T11:43:00+01:00Tadej Janežtag:tadej.ja.nez.si,2017-01-13:/dont-mix-yum-dnf-and-pip.html
<p>Too many times I've seen people using a mix of <a href="http://yum.baseurl.org/">yum</a>/
<a href="http://dnf.baseurl.org/">dnf</a> and <a href="https://pip.pypa.io/">pip</a> for
installation of system-wide Python packages This causes <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#problems-with-mixing-yumdnf-and-pip">all sorts of problems
</a>.</p>
<p><strong>TL; DR</strong> <em>Install system-wide Python packages with yum/dnf and only use pip
inside a <a href="https://virtualenv.pypa.io/">virtual environment</a>. For an example,
see my <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#ansible-playbook-for-installation-of-packages-inside-a-virtual-environment">Ansible snippet below
</a>.</em></p>
<p><em>Update (January 25, 2017): Updated <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#proper-way-to-to-use-pip-without-mixing-it-with-yumdnf">instructions on preparing the Python
virtual environment to work with setuptools 34+ on CentOS/RHEL 7</a>.
Also updated <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#ansible-playbook-for-installation-of-packages-inside-a-virtual-environment">the Ansible playbook</a>
to skip seperately updating pip and setuptools upon creation of a new virtual
environment on Fedora.</em></p>
<p>Too many times I've seen people using a mix of <a href="http://yum.baseurl.org/">yum</a>/
<a href="http://dnf.baseurl.org/">dnf</a> and <a href="https://pip.pypa.io/">pip</a> for
installation of system-wide Python packages This causes <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#problems-with-mixing-yumdnf-and-pip">all sorts of problems
</a>.</p>
<p><strong>TL; DR</strong> <em>Install system-wide Python packages with yum/dnf and only use pip
inside a <a href="https://virtualenv.pypa.io/">virtual environment</a>. For an example,
see my <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#ansible-playbook-for-installation-of-packages-inside-a-virtual-environment">Ansible snippet below
</a>.</em></p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>I'll use <a href="https://www.centos.org/">CentOS</a> 7 for the examples, but the
concepts apply to all current <a href="https://getfedora.org/">Fedora</a> and
CentOS/<a href="https://www.redhat.com/en/technologies/linux-platforms/enterprise-linux">RHEL</a>
distributions.</p>
</div>
<h2 id="problems-with-mixing-yumdnf-and-pip">Problems with mixing yum/dnf and pip</h2>
<p>People usually install the <code>python2-pip</code> package to boot-strap installation of
pip on their systems:</p>
<div class="highlight"><pre><span></span>sudo yum -y install epel-release
sudo yum -y install python2-pip
</pre></div>
<p>At the time of writing, this will install pip 8.1.2 on the system, while the
latest pip version is 9.0.1.
A user will soon notice that on each run of the <code>pip</code> command the following
message is shown:</p>
<div class="highlight"><pre><span></span>You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
</pre></div>
<p>Naively, he will upgrade pip with:</p>
<div class="highlight"><pre><span></span>sudo pip install --upgrade pip
</pre></div>
<p>which will replace the previous pip installation in system locations (e.g.
<code>/usr/bin</code>, <code>/usr/lib/python2.7/site-packages/pip</code>, ...) with the new pip
version.</p>
<p>The problem is that this will leave <a href="http://rpm.org/">RPM</a> with no clue as to
what is going on. If you run <code>rpm --verify python2-pip</code> it will show that
everything is broken:</p>
<div class="highlight"><pre><span></span>S.5....T. /usr/bin/pip
S.5....T. /usr/bin/pip2
S.5....T. /usr/bin/pip2.7
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/PKG-INFO
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/SOURCES.txt
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/dependency_links.txt
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/entry_points.txt
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/not-zip-safe
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/requires.txt
missing /usr/lib/python2.7/site-packages/pip-8.1.2-py2.7.egg-info/top_level.txt
S.5....T. /usr/lib/python2.7/site-packages/pip/__init__.py
S.5....T. /usr/lib/python2.7/site-packages/pip/__init__.pyc
missing /usr/lib/python2.7/site-packages/pip/__init__.pyo
.......T. /usr/lib/python2.7/site-packages/pip/__main__.py
S.5....T. /usr/lib/python2.7/site-packages/pip/__main__.pyc
missing /usr/lib/python2.7/site-packages/pip/__main__.pyo
<span class="o">[</span> ... output trimmed ... <span class="o">]</span>
S.5....T. /usr/lib/python2.7/site-packages/pip/vcs/subversion.py
S.5....T. /usr/lib/python2.7/site-packages/pip/vcs/subversion.pyc
missing /usr/lib/python2.7/site-packages/pip/vcs/subversion.pyo
S.5....T. /usr/lib/python2.7/site-packages/pip/wheel.py
S.5....T. /usr/lib/python2.7/site-packages/pip/wheel.pyc
missing /usr/lib/python2.7/site-packages/pip/wheel.pyo
</pre></div>
<p>Furthermore, when a new version of the <code>python2-pip</code> becomes available, yum
upgrade will complain about not being able to find some files and directories.</p>
<p>For example, on upgrade from <code>python-pip-7.1.0-1.el7</code> to
<code>python2-pip-8.1.2-5.el7</code> when pip was upgraded with pip in-between, yum
upgrade gives the following warnings:</p>
<div class="highlight"><pre><span></span>warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/markers.pyo: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/markers.pyc: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/markers.py: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/__init__.pyo: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/__init__.pyc: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib/__init__.py: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip/_vendor/_markerlib: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/top_level.txt: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/requires.txt: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/pbr.json: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/not-zip-safe: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/entry_points.txt: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/dependency_links.txt: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/SOURCES.txt: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info/PKG-INFO: remove failed: No such file or directory
warning: file /usr/lib/python2.7/site-packages/pip-7.1.0-py2.7.egg-info: remove failed: No such file or directory
</pre></div>
<h2 id="proper-way-to-to-use-pip-without-mixing-it-with-yumdnf">Proper way to to use pip without mixing it with yum/dnf</h2>
<p>The solution is to limit pip's use to installation of Python packages inside a
Python virtual environment.</p>
<p>First install the <code>python-virtualenv</code> package with yum/dnf:</p>
<div class="highlight"><pre><span></span>sudo yum -y install python-virtualenv
</pre></div>
<p>Then use it to create and activate a new virtual environment:</p>
<div class="highlight"><pre><span></span>virtualenv myvenv
source myvenv/bin/activate
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>At the time of writing, CentOS/RHEL 7's system-installed virtualenv package
is very old (1.10.1) and creates a virtual environment with very old pip
(1.4.1) and <a href="https://setuptools.readthedocs.io/">setuptools</a> (0.9.8)
packages. Hence, it is recommended to update them separately, before
installation of other things in the virtual environment.</p>
<p>Since <a href="https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0">setuptools version 34</a>,
setuptools package no longer bundles its requirements and relies on
installing whell distributions of its requirements. To install these
wheels, it needs a newer version pip, hence pip needs to be updated
separately before setuptools:</p>
<div class="highlight"><pre><span></span>pip install -U pip
pip install -U setuptools
</pre></div>
<p>A complete Ansible playbook for proper creation of a Python virtual
environment is provided <a href="https://tadej.ja.nez.si/dont-mix-yum-dnf-and-pip.html#ansible-playbook-for-installation-of-packages-inside-a-virtual-environment">later</a>.</p>
</div>
<p>Then install whichever Python package you want inside the Python virtual
environment.</p>
<h3 id="tip-dont-install-the-system-pip-package">Tip: Don't install the system pip package</h3>
<p>To avoid mistakenly using pip outside a virtual environment, don't install
the pip system package (e.g. <code>python2-pip</code>).</p>
<p>This way, <code>pip</code> executable will only be available after you activate the chosen
virtual environment, which limits mistakes with using the system-wide pip to
the minimum.</p>
<h2 id="ansible-playbook-for-installation-of-packages-inside-a-virtual-environment">Ansible playbook for installation of packages inside a virtual environment</h2>
<p>If you need to automate installation of <code>python-virtualenv</code> system package and
creation of a Python virtual environment for installation of project's
requirements inside this virtual environment, you can use the following
<a href="https://www.ansible.com/">Ansible</a> playbook:</p>
<div class="highlight"><pre><span></span><span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Playbook for installation of packages inside a virtual environment</span>
<span class="l l-Scalar l-Scalar-Plain">hosts</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">all</span>
<span class="l l-Scalar l-Scalar-Plain">become</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">true</span>
<span class="l l-Scalar l-Scalar-Plain">vars</span><span class="p p-Indicator">:</span>
<span class="c1"># adjust these variables to your project's needs</span>
<span class="l l-Scalar l-Scalar-Plain">venv_path</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">/opt/myvenv/</span>
<span class="l l-Scalar l-Scalar-Plain">requirements</span><span class="p p-Indicator">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">requests</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">simplejson</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">six</span>
<span class="l l-Scalar l-Scalar-Plain">tasks</span><span class="p p-Indicator">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Install python-virtualenv package</span>
<span class="l l-Scalar l-Scalar-Plain">package</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">name=python-virtualenv state=installed</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">block</span><span class="p p-Indicator">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Create virtual environment with up-to-date pip</span>
<span class="l l-Scalar l-Scalar-Plain">pip</span><span class="p p-Indicator">:</span>
<span class="l l-Scalar l-Scalar-Plain">virtualenv</span><span class="p p-Indicator">:</span> <span class="s">"{{</span><span class="nv"> </span><span class="s">venv_path</span><span class="nv"> </span><span class="s">}}"</span>
<span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">pip</span>
<span class="l l-Scalar l-Scalar-Plain">state</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">latest</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Update virtual environment's setuptools</span>
<span class="l l-Scalar l-Scalar-Plain">pip</span><span class="p p-Indicator">:</span>
<span class="l l-Scalar l-Scalar-Plain">virtualenv</span><span class="p p-Indicator">:</span> <span class="s">"{{</span><span class="nv"> </span><span class="s">venv_path</span><span class="nv"> </span><span class="s">}}"</span>
<span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">setuptools</span>
<span class="l l-Scalar l-Scalar-Plain">state</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">latest</span>
<span class="l l-Scalar l-Scalar-Plain">when</span><span class="p p-Indicator">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">ansible_distribution in ["CentOS", "RedHat"]</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">ansible_distribution_major_version | int == 7</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Install project's requirements in virtual environement</span>
<span class="l l-Scalar l-Scalar-Plain">pip</span><span class="p p-Indicator">:</span>
<span class="l l-Scalar l-Scalar-Plain">virtualenv</span><span class="p p-Indicator">:</span> <span class="s">"{{</span><span class="nv"> </span><span class="s">venv_path</span><span class="nv"> </span><span class="s">}}"</span>
<span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="s">"{{</span><span class="nv"> </span><span class="s">requirements</span><span class="nv"> </span><span class="s">}}"</span>
<span class="l l-Scalar l-Scalar-Plain">state</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">latest</span>
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Due to a <a href="https://github.com/ansible/ansible-modules-core/issues/5347">bug in Ansible's <code>pip</code> module</a> which causes
Ansible to use system-installed <code>pip2</code> executable and ignore the <code>pip</code>
executable installed inside the virtual environment, you must use at least
<a href="https://releases.ansible.com/ansible/ansible-2.2.1.0-0.3.rc3.tar.gz">version 2.2.1 RC3</a> which
fixes the bug.</p>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Fedora comes with <a href="https://apps.fedoraproject.org/packages/python-virtualenv">virtualenv 14.0.1+</a> which
<a href="http://virtualenv.readthedocs.io/en/stable/changes/#id11">automatically downloads new releases of pip, setuptools, wheel and their
requirements from PyPI</a>, therefore there
is no need to update them separately before installing project's
requirements in virtual environment.</p>
</div>
<p>If your project's requirements are not pure Python packages, but also include
packages with <a href="https://docs.python.org/3/extending/index.html">C/C++ extensions</a>, they'll need to be built as
part of the installation process. Therefore, you'll need to have at least <code>gcc</code>
and <code>python-devel</code> packages installed. You can use the following Ansible task
to achieve that:</p>
<div class="highlight"><pre><span></span><span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">name</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">Install project's building prerequisites</span>
<span class="l l-Scalar l-Scalar-Plain">package</span><span class="p p-Indicator">:</span> <span class="l l-Scalar l-Scalar-Plain">name={{ item }} state=installed</span>
<span class="l l-Scalar l-Scalar-Plain">with_items</span><span class="p p-Indicator">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">gcc</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">python-devel</span>
<span class="c1"># add other packages required to build your project's requirements</span>
</pre></div>
<p>Put it before the <em>Install project's requirements in virtual environement</em>
task.</p>Publishing a Pelican site to GitHub Pages using Fabric2016-10-19T19:05:00+02:002016-10-19T19:05:00+02:00Tadej Janežtag:tadej.ja.nez.si,2016-10-19:/publishing-to-github-pages.html
<p>In the <a href="https://tadej.ja.nez.si/setting-up-pelican-site.html">previous blog post</a> we looked at
setting up a <a href="http://getpelican.com/">Pelican</a> site with <a href="https://www.python.org/">Python 3</a> and <a href="http://www.fabfile.org/">Fabric</a> on <a href="https://getfedora.org/">Fedora 24</a>.</p>
<p>Now that we have a Pelican site up and running, I will show you how to publish
it on <a href="https://pages.github.com/">GitHub Pages</a> using a sleek Fabric task.</p>
<p><em>Update (May 25, 2018): Starting with May 1, 2018, <a href="https://blog.github.com/2018-05-01-github-pages-custom-domains-https/">GitHub Pages gained support
for serving custom domains over HTTPS</a>.
I've updated the <a href="#setting-up-a-custom-domain">Setting up a custom domain</a>
section of this blog post to advise users to use this functionality.</em></p>
<p>In the <a href="https://tadej.ja.nez.si/setting-up-pelican-site.html">previous blog post</a> we looked at
setting up a <a href="http://getpelican.com/">Pelican</a> site with <a href="https://www.python.org/">Python 3</a> and <a href="http://www.fabfile.org/">Fabric</a> on <a href="https://getfedora.org/">Fedora 24</a>.</p>
<p>Now that we have a Pelican site up and running, I will show you how to publish
it on <a href="https://pages.github.com/">GitHub Pages</a> using a sleek Fabric task.</p>
<h2 id="review-of-existing-approaches">Review of existing approaches</h2>
<p>There are various approaches to managing publishing your Pelican site to
GitHub Pages.</p>
<p><a href="http://docs.getpelican.com/en/stable/tips.html#publishing-to-github">Pelican authors recommend</a> using the
<a href="https://github.com/davisp/ghp-import"><code>ghp-import</code> tool</a> to import the
contents of the <code>output</code> directory to a special <code>gh-pages</code> git branch, which
can then be pushed to the desired GitHub pages repository's branch (i.e.
<code>master</code> branch for User pages or <code>gh-pages</code> branch for Project pages).</p>
<p><a href="http://ankursinha.in/blog/">Ankur Sinha</a> wrote an
<a href="https://fedoramagazine.org/make-github-pages-blog-with-pelican/">excellent article for Fedora Magazine</a>, where he
recommends creating two git repositories, the main repository containing the
source of the Pelican page and the second repository containing the contents of
the <code>output</code> directory.
The recommendation is to add the second repository as a submodule of the main
repository.</p>
<p>I didn't find any of those two approaches satisfy my needs.
The downside of using the <code>ghp-import</code> tool is that it is not packaged for
Fedora yet (<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1183422">review request</a>) and that it destroys the
<code>gh-pages</code> branch on each run, thus one is unable to keep previous contents of
the page as older commits.
On the other hand, tracking the contents of the <code>output</code> directory as a
submodule in a separate git repo has the disadvantage of having to update the
submodule reference in the main git repo every time a new version of the site
is built, thus leading to a large number of "submodule bump" commits in the
main git repo.</p>
<h2 id="a-new-approach-using-a-custom-fabric-task">A new approach using a custom Fabric task</h2>
<p>Therefore, I crafted a new approach that tries to overcome these disadvantages.
Like in Ankur Sinha's article, I created two git repositories, the main
repository containing the source of the Pelican page and the second repository
containing the contents of the <code>output</code> directory.
However, instead of linking the repositories via a submodule, I just created
a custom Fabric task that rebuilds the source page, commits its output to the
second git repository and pushes it to GitHub Pages.</p>
<p>To use this approach, first create the <em>username</em>.github.io repository on
<a href="https://github.com/new">GitHub</a>.</p>
<p>Then generate a clean production version of the site:</p>
<div class="highlight"><pre><span></span>fab clean
fab preview
</pre></div>
<p>Add the contents of the <code>output</code> directory to the <em>username</em>.github.io git
repository:</p>
<div class="highlight"><pre><span></span>cd output
git init
git add --all
git commit -m "Initial commit"
git remote add origin git@github.com:<username>/<username>.github.io.git
git push origin master
</pre></div>
<p>Edit <code>publishconf.py</code> and remove the following line:</p>
<div class="highlight"><pre><span></span><span class="n">DELETE_OUTPUT_DIRECTORY</span> <span class="o">=</span> <span class="bp">True</span>
</pre></div>
<p>This will prevent Pelican from deleting the whole <code>output</code> directory, including
the git repository initialized in the previous step, when building the
production version of the site.</p>
<p>Modify the <code>clean</code> function in Fabric's <code>fabfile.py</code> to not delete the <code>.git</code>
directory:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">clean</span><span class="p">():</span>
<span class="sd">"""Remove generated files"""</span>
<span class="k">for</span> <span class="n">root</span><span class="p">,</span> <span class="n">dirs</span><span class="p">,</span> <span class="n">files</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">walk</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">):</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">dirs</span><span class="p">[:]:</span>
<span class="c1"># Do not recurse into this directory</span>
<span class="n">dirs</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">if</span> <span class="n">name</span> <span class="o">==</span> <span class="s1">'.git'</span><span class="p">:</span>
<span class="c1"># Do not remove .git/ directory</span>
<span class="k">pass</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">name</span><span class="p">))</span>
<span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">files</span><span class="p">:</span>
<span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="n">name</span><span class="p">))</span>
</pre></div>
<p>Add the following <code>gh_pages</code> function (and its <code>publish</code> alias) to Fabric's
<code>fabfile.py</code>:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">fabric.contrib.console</span> <span class="kn">import</span> <span class="n">confirm</span>
<span class="c1"># Get absolute path of project's root directory</span>
<span class="n">env</span><span class="o">.</span><span class="n">project_root</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">real_fabfile</span><span class="p">)</span>
<span class="c1"># Set absolute path of project's deploy directory</span>
<span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">project_root</span><span class="p">,</span> <span class="s1">'output'</span><span class="p">)</span>
<span class="c1"># Github Pages configuration</span>
<span class="n">env</span><span class="o">.</span><span class="n">github_pages_branch</span> <span class="o">=</span> <span class="s1">'master'</span>
<span class="k">def</span> <span class="nf">gh_pages</span><span class="p">():</span>
<span class="sd">"""Publish to GitHub Pages"""</span>
<span class="k">with</span> <span class="n">lcd</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">project_root</span><span class="p">):</span>
<span class="c1"># ensure the main git repository is clean</span>
<span class="n">main_git_unclean</span> <span class="o">=</span> <span class="n">local</span><span class="p">(</span><span class="s1">'git status --untracked-files=no --porcelain'</span><span class="p">,</span>
<span class="n">capture</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">if</span> <span class="n">main_git_unclean</span><span class="p">:</span>
<span class="n">abort</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="s2">"The main git repository is not clean:"</span><span class="p">,</span>
<span class="n">main_git_unclean</span><span class="p">]))</span>
<span class="c1"># get main git repository's HEAD's sha checksum</span>
<span class="n">main_commit_sha</span> <span class="o">=</span> <span class="n">local</span><span class="p">(</span><span class="s1">'git rev-parse --short HEAD'</span><span class="p">,</span> <span class="n">capture</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">with</span> <span class="n">lcd</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">):</span>
<span class="c1"># sync local GitHub Pages git repository with remote repository</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git fetch origin {github_pages_branch}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">**</span><span class="n">env</span><span class="p">))</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git reset --hard origin/{github_pages_branch}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">**</span><span class="n">env</span><span class="p">))</span>
<span class="n">clean</span><span class="p">()</span>
<span class="c1"># build a production version of the site</span>
<span class="n">preview</span><span class="p">()</span>
<span class="k">with</span> <span class="n">lcd</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">):</span>
<span class="n">pages_git_unclean</span> <span class="o">=</span> <span class="n">local</span><span class="p">(</span><span class="s1">'git status --porcelain'</span><span class="p">,</span> <span class="n">capture</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">if</span> <span class="n">pages_git_unclean</span><span class="p">:</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git add --all'</span><span class="p">)</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git commit -m "Build of source repo @ {}"'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">main_commit_sha</span><span class="p">))</span>
<span class="k">if</span> <span class="n">confirm</span><span class="p">(</span><span class="s2">"Do you wish to publish the current version of the "</span>
<span class="s2">"page to GitHub Pages?"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git push origin {github_pages_branch}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">**</span><span class="n">env</span><span class="p">))</span>
<span class="n">commit_sha</span> <span class="o">=</span> <span class="n">local</span><span class="p">(</span><span class="s1">'git rev-parse --short HEAD'</span><span class="p">,</span> <span class="n">capture</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">puts</span><span class="p">(</span><span class="s2">"Pushed commit {} to GitHub Pages"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">commit_sha</span><span class="p">))</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># reset the git repo to the one on GitHub Pages</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'git reset origin/master'</span><span class="p">)</span>
<span class="n">puts</span><span class="p">(</span><span class="s2">"Exiting on user request."</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">puts</span><span class="p">(</span><span class="s2">"Nothing has changed. Exiting."</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">publish</span><span class="p">():</span>
<span class="sd">"""Publish to GitHub Pages"""</span>
<span class="n">gh_pages</span><span class="p">()</span>
</pre></div>
<p>To publish the page, run:</p>
<div class="highlight"><pre><span></span>fab publish
</pre></div>
<p>and answer <code>y</code> when prompted.</p>
<p>Visit the page at https://<em>username</em>.github.io and share it with the World!</p>
<h2 id="setting-up-a-custom-domain">Setting up a custom domain</h2>
<p>To use a custom domain with GitHub Pages, e.g. <code>blog.mysite.com</code>, one needs to
add a <code>CNAME</code> file in the root of the generated site which tells GitHub Pages
on which custom domain the site is hosted.</p>
<p>The contents of the file should match the custom domain name. For the example
custom domain above, one would create the file <code>content/extra/CNAME</code> with
the following content:</p>
<div class="highlight"><pre><span></span>blog.mysite.com
</pre></div>
<p>To instruct Pelican to copy the <code>CNAME</code> file to site's root, list it among
site's static paths:</p>
<div class="highlight"><pre><span></span><span class="n">STATIC_PATHS</span> <span class="o">=</span> <span class="p">[</span>
<span class="o">...</span> <span class="n">other</span> <span class="n">static</span> <span class="n">paths</span> <span class="o">...</span>
<span class="c1"># GitHub Pages custom domain</span>
<span class="s1">'extra/CNAME'</span><span class="p">,</span>
<span class="p">]</span>
</pre></div>
<p>and annotate it with extra path metadata:</p>
<div class="highlight"><pre><span></span><span class="n">EXTRA_PATH_METADATA</span> <span class="o">=</span> <span class="p">{</span>
<span class="o">...</span> <span class="n">other</span> <span class="n">extra</span> <span class="n">path</span> <span class="n">metadata</span> <span class="o">...</span>
<span class="s1">'extra/CNAME'</span><span class="p">:</span> <span class="p">{</span><span class="s1">'path'</span><span class="p">:</span> <span class="s1">'CNAME'</span><span class="p">},</span>
<span class="p">}</span>
</pre></div>
<p>Finally, change the value of <code>SITEURL</code> variable in <code>publishconf.py</code> to the
custom domain's name and commit the changes:</p>
<div class="highlight"><pre><span></span>git add content/extra/CNAME
git commit -a -m "Change site's URL to https://blog.mysite.com"
</pre></div>
<p>Before we publish the new version of the site with a custom domain, we need
to configure an appropriate DNS record with our DNS provider. The above example
custom domain is a custom <strong>subdomain</strong>, so we need to set up a <code>CNAME</code> record
with our DNS provider that points to <em>username</em>.github.io. Follow your DNS
provider's instructions on how to do that.</p>
<p>To confirm that the new DNS record is set up correctly, use the <code>dig</code> utility:</p>
<div class="highlight"><pre><span></span>dig +nocmd +nostats +nocomments blog.mysite.com
</pre></div>
<p>and make sure the output is similar to:</p>
<div class="highlight"><pre><span></span>;blog.mysite.com. IN A
blog.mysite.com. 1747 IN CNAME username.github.io.
username.github.io. 3547 IN CNAME github.map.fastly.net.
github.map.fastly.net. 650 IN CNAME prod.github.map.fastlylb.net.
prod.github.map.fastlylb.net. 17 IN A 151.101.12.133
</pre></div>
<p>If your custom domain is an apex domain (e.g. <code>mysite.com</code>), you will need
to configure a different type of a DNS record. See <a href="https://help.github.com/articles/setting-up-an-apex-domain/">GitHub Pages's Help on
Setting up an apex domain</a>.</p>
<p>After DNS is properly configured, publish the updated site to GitHub Pages
with:</p>
<div class="highlight"><pre><span></span>fab publish
</pre></div>
<p>Visit the page at your custom domain!</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Starting with May 1, 2018, <a href="https://blog.github.com/2018-05-01-github-pages-custom-domains-https/">GitHub Pages gained support for serving custom
domains over HTTPS</a>.</p>
<p>You are advised to take advantage of this functionality and server your
site over HTTPS. If you've used a <code>CNAME</code> record with your DNS provider,
then you should be all set.</p>
<p>After ensuring your site loads correctly over HTTPS, browse to
https://github.com/<em>username</em>/<em>username</em>.github.io/settings and
enable the <em>Enforce HTTPS</em> option under the <em>GitHub Pages</em> section. This
will transparently redirect any HTTP requests to your site to HTTPS.</p>
</div>
<p>To see this implemented in practice, browse the
<a href="https://github.com/tjanez/site/tree/5012f31">source repo of my Pelican site</a>.</p>Setting up a Pelican site with Python 3 and Fabric on Fedora 242016-10-08T10:16:00+02:002016-10-08T10:16:00+02:00Tadej Janežtag:tadej.ja.nez.si,2016-10-08:/setting-up-pelican-site.html
<p>According to <a href="https://www.staticgen.com/">StaticGen</a>, <a href="http://getpelican.com/">Pelican</a> is the most popular static site generator written in
<a href="https://www.python.org/">Python</a>. As of Oct 10, 2016 it has
<a href="https://github.com/getpelican/pelican">6168 stars on GitHub</a>.</p>
<p>In this blog post, I'll show you how to create your own Pelican site, track
it in a <a href="https://git-scm.com/">git</a> repository, use <a href="http://www.fabfile.org/">Fabric</a> to administer it, change site's default theme and
finally, create a Hello World blog post.</p>
<p><em>Update (December 6, 2016): Newer Fedora Pelican packages (
<a href="http://koji.fedoraproject.org/koji/search?terms=python-pelican-3.6.3-6.fc24&type=build&match=glob">3.6.3-6.fc24</a>,
<a href="http://koji.fedoraproject.org/koji/search?terms=python-pelican-3.6.3-6.fc25&type=build&match=glob">3.6.3-6.fc24</a>,
<a href="http://koji.fedoraproject.org/koji/search?terms=python-pelican-3.6.3-6.fc26&type=build&match=glob">3.6.3-6.fc26</a>
) <a href="https://pkgs.fedoraproject.org/cgit/rpms/python-pelican.git/commit/?id=998b43e24184da9293815a5b4e965d6d954cf1a2">changed naming of Python 3 Pelican executables</a> from
<code>py3-<command></code> to <code><command>-3</code> so make sure to update to the latest Fedora
Pelican packages and use the <code><command>-3</code> syntax.</em></p>
<p><em>Update (October 19, 2016):
The <a href="https://tadej.ja.nez.si/publishing-to-github-pages.html">follow-up blog post</a> on
publishing your site to GitHub Pages with a sleek Fabric task has been
published.</em></p>
<p>According to <a href="https://www.staticgen.com/">StaticGen</a>, <a href="http://getpelican.com/">Pelican</a> is the most popular static site generator written in
<a href="https://www.python.org/">Python</a>. As of Oct 10, 2016 it has
<a href="https://github.com/getpelican/pelican">6168 stars on GitHub</a>.</p>
<p>In this blog post, I'll show you how to create your own Pelican site, track
it in a <a href="https://git-scm.com/">git</a> repository, use <a href="http://www.fabfile.org/">Fabric</a> to administer it, change site's default theme and
finally, create a Hello World blog post.</p>
<h2 id="installing-prerequisites">Installing prerequisites</h2>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>This example uses a vanilla <a href="https://getfedora.org/en/cloud/">Fedora 24 Cloud</a> system.</p>
</div>
<p>Install Pelican (Python 3 version), git and Fabric (unfortunately,
<a href="http://fedora.portingdb.xyz/pkg/fabric/">only Python 2 version is currently available</a>):</p>
<div class="highlight"><pre><span></span>sudo dnf -y install python3-pelican git fabric
</pre></div>
<h2 id="creating-a-git-repository-for-the-site">Creating a git repository for the site</h2>
<p>Create a directory for the site:</p>
<div class="highlight"><pre><span></span>mkdir my-site
cd my-site
</pre></div>
<p>To setup a Pelican skeleton for the site, run <code>pelican-quickstart-3</code> and
answer the questions. If you are unsure, you can safely accept the default
answer.</p>
<p>Before initializing the git repository, clean up the generated skeleton.
Remove the <code>Makefile</code> and edit <code>fabfile.py</code> to remove the unnecessary
functionality and make it work with Python 3 version of Pelican.</p>
<p>All <code>pelican</code> commands need to be replaced with <code>pelican-3</code>. In addition,
the <code>serve()</code> function needs to be rewritten since it tries to directly import
the Python 2 version of <code>pelican.server</code> which is not available. You can also
safely remove the parts connected with Rackspace Cloud Files, rsync publishing
and <a href="https://pages.github.com/">GitHub Pages</a> (I'll describe how to add support
for it in a follow-up blog post).</p>
<p>The cleaned-up version of <code>fabfile.py</code> should look something like:</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">fabric.api</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">import</span> <span class="nn">fabric.contrib.project</span> <span class="kn">as</span> <span class="nn">project</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">shutil</span>
<span class="c1"># Local path configuration (can be absolute or relative to fabfile)</span>
<span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span> <span class="o">=</span> <span class="s1">'output'</span>
<span class="c1"># Port for `serve`</span>
<span class="n">PORT</span> <span class="o">=</span> <span class="mi">8000</span>
<span class="k">def</span> <span class="nf">clean</span><span class="p">():</span>
<span class="sd">"""Remove generated files"""</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">):</span>
<span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">build</span><span class="p">():</span>
<span class="sd">"""Build local version of site"""</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'pelican-3 -s pelicanconf.py'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">rebuild</span><span class="p">():</span>
<span class="sd">"""`clean`, then `build`"""</span>
<span class="n">clean</span><span class="p">()</span>
<span class="n">build</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">regenerate</span><span class="p">():</span>
<span class="sd">"""Automatically regenerate site upon file modification"""</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'pelican-3 -r -s pelicanconf.py'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">serve</span><span class="p">():</span>
<span class="sd">"""Serve site at http://localhost:PORT/"""</span>
<span class="k">with</span> <span class="n">lcd</span><span class="p">(</span><span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span><span class="p">):</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'python3 -m pelican.server {}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">PORT</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">reserve</span><span class="p">():</span>
<span class="sd">"""`build`, then `serve`"""</span>
<span class="n">build</span><span class="p">()</span>
<span class="n">serve</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">preview</span><span class="p">():</span>
<span class="sd">"""Build production version of site"""</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'pelican-3 -s publishconf.py'</span><span class="p">)</span>
</pre></div>
<p>Since we are using the Python 3 version of Pelican, we can remove the Python 2
compatibility headers from <code>pelicanconf.py</code> and <code>publishconf.py</code>:</p>
<div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*- #</span>
<span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">unicode_literals</span>
</pre></div>
<p>In addition, remove the shebang from <code>publishconf.py</code>.</p>
<p>Initialize the git repository and create the initial commit:</p>
<div class="highlight"><pre><span></span>git init
git add *.py
git commit -m "Initial site created with pelican-quickstart"
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If you use git for the first time, you must configure your git email and
user name with:</p>
<div class="highlight"><pre><span></span>git config --global user.email <span class="s2">"&lt;your-email-address&gt;"</span>
git config --global user.name <span class="s2">"&lt;your-name&gt;"</span>
</pre></div>
</div>
<p>To instruct git to ignore the generated Python byte-code and the generated
site, create <code>.gitignore</code> file with the following contents:</p>
<div class="highlight"><pre><span></span># ignore Python byte-code
*.pyc
# ignore generated site
/output/
</pre></div>
<p>and commit it with:</p>
<div class="highlight"><pre><span></span>git add .gitignore
git commit -m "Add .gitignore"
</pre></div>
<h2 id="administering-the-site-with-fabric">Administering the site with Fabric</h2>
<p>Fabric is a Python library and a command-line tool for automating deployment
and system administration tasks. By default, it looks for a <code>fabfile.py</code> file
where one can define Fabric's tasks. See the <code>fabfile.py</code> code listing above
to get a glimpse of how Fabric tasks look like.</p>
<p>To run Fabric tasks, just execute <code>fab</code> followed by the task's name, e.g.
<code>serve</code>. Here are a couple of tasks that you will typically use when
administering a site.</p>
<p>To generate the site, use:</p>
<div class="highlight"><pre><span></span>fab build
</pre></div>
<p>To serve the site locally on port 8080, use:</p>
<div class="highlight"><pre><span></span>fab serve
</pre></div>
<p>To regenerate the site and serve it locally, use:</p>
<div class="highlight"><pre><span></span>fab reserve
</pre></div>
<p>To automatically regenerate the site upon file modification and serve it
locally, run the following commands in two separate terminals:</p>
<div class="highlight"><pre><span></span>fab regenerate
fab serve
</pre></div>
<h2 id="changing-sites-default-theme">Changing site's default theme</h2>
<p>Frankly speaking, the default Pelican theme looks dated nowadays, so you'll
want to change it sooner rather than later.
Take a look at the <a href="http://www.pelicanthemes.com/">Pelican Themes site</a> and
find a theme you like.</p>
<p>After you decide which theme you'll use (in the example I'll use <a href="https://github.com/alexandrevicenzi/Flex/">Alexandre
Vicenzi's Flex theme</a>, which I use
for my Pelican site), add it to your git repo as a submodule:</p>
<div class="highlight"><pre><span></span>git submodule add https://github.com/alexandrevicenzi/Flex.git Flex
</pre></div>
<p>Then configure it in <code>pelicanconf.py</code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Each theme has its own configuration. Consult the chosen theme's
documentation on what you can configure.</p>
</div>
<p>Here is an example configuration for the Flex theme:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">datetime</span>
<span class="kn">import</span> <span class="nn">hashlib</span>
<span class="o">...</span> <span class="n">other</span> <span class="n">configuration</span> <span class="o">...</span>
<span class="c1"># Theme</span>
<span class="n">THEME</span> <span class="o">=</span> <span class="s1">'Flex'</span>
<span class="n">SITETITLE</span> <span class="o">=</span> <span class="n">SITENAME</span>
<span class="n">SITESUBTITLE</span> <span class="o">=</span> <span class="s1">'My cool descrition'</span>
<span class="n">email</span> <span class="o">=</span> <span class="s1">'my.email@somedomain.com'</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">'utf-8'</span><span class="p">)</span>
<span class="n">SITELOGO</span> <span class="o">=</span> <span class="s1">'https://seccdn.libravatar.org/avatar/{}?s=256'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">hashlib</span><span class="o">.</span><span class="n">md5</span><span class="p">(</span><span class="n">email</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="o">.</span><span class="n">lower</span><span class="p">())</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">())</span>
<span class="n">copyright_year_start</span> <span class="o">=</span> <span class="mi">2016</span>
<span class="n">copyright_year_end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="o">.</span><span class="n">today</span><span class="p">()</span><span class="o">.</span><span class="n">year</span>
<span class="k">if</span> <span class="n">copyright_year_end</span> <span class="o">==</span> <span class="n">copyright_year_start</span><span class="p">:</span>
<span class="n">COPYRIGHT_YEAR</span> <span class="o">=</span> <span class="n">copyright_year_start</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">COPYRIGHT_YEAR</span> <span class="o">=</span> <span class="s1">'{}-{}'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">copyright_year_start</span><span class="p">,</span> <span class="n">copyright_year_end</span><span class="p">)</span>
<span class="n">PYGMENTS_STYLE</span> <span class="o">=</span> <span class="s1">'native'</span>
</pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>In the example, I use <a href="https://www.libravatar.org/">libravatar</a>, a
federated open source avatar hosting service, for my site's logo.
To use it for your own site, <a href="https://www.libravatar.org/account/new/">create an account</a> with them.</p>
</div>
<p>Finally, commit the changes to git:</p>
<div class="highlight"><pre><span></span>git commit -a -m "Use Flex theme"
</pre></div>
<h2 id="creating-a-hello-world-blog-post">Creating a Hello World blog post</h2>
<p>To create your first blog post, create a Markdown file in the <code>content</code>
directory with the following content:</p>
<div class="highlight"><pre><span></span><span class="n">Title</span><span class="o">:</span> <span class="n">Hello</span><span class="o">,</span> <span class="n">World</span><span class="o">!</span>
<span class="n">Date</span><span class="o">:</span> <span class="mi">2016</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="mi">06</span> <span class="mi">10</span><span class="o">:</span><span class="mi">52</span>
<span class="n">Category</span><span class="o">:</span> <span class="n">Site</span>
<span class="n">This</span> <span class="k">is</span> <span class="n">my</span> <span class="n">first</span> <span class="n">blog</span> <span class="n">post</span> <span class="n">using</span> <span class="o">[</span><span class="n">Pelican</span><span class="o">](</span><span class="n">http</span><span class="o">://</span><span class="n">blog</span><span class="o">.</span><span class="na">getpelican</span><span class="o">.</span><span class="na">com</span><span class="o">/)</span>
<span class="n">and</span> <span class="o">[</span><span class="n">Flex</span><span class="o">](</span><span class="n">https</span><span class="o">://</span><span class="n">github</span><span class="o">.</span><span class="na">com</span><span class="sr">/alexandrevicenzi/Flex/</span><span class="o">)!</span>
</pre></div>
<p>Commit the changes to git with:</p>
<div class="highlight"><pre><span></span>git add content/
git commit -m "Add Hello World blog post"
</pre></div>
<p>Preview the site with:</p>
<div class="highlight"><pre><span></span>fab reserve
</pre></div>
<h2 id="next-steps">Next steps</h2>
<p>You have successfully completed setting up a Pelican site. But the site doesn't
really serve its purpose if its only available on your local computer, does it?</p>
<p>I plan to write a follow-up blog post that will show you how to publish your
site to <a href="https://pages.github.com/">GitHub Pages</a> with a sleek Fabric task to
do it automatically. Stay tuned!</p>
<p>Meanwhile, you can also browse the <a href="https://github.com/tjanez/site">source repo of my Pelican site</a>.</p>Hello, World!2016-10-06T10:52:00+02:002016-10-06T10:52:00+02:00Tadej Janežtag:tadej.ja.nez.si,2016-10-06:/hello-world.html<p>I finally made a personal web site with <a href="http://blog.getpelican.com/">Pelican</a>
and Alexandre Vicenzi's great <a href="https://github.com/alexandrevicenzi/Flex/">Flex</a>
theme :)!</p>
<p>Stay tuned for a follow-up blog post, where I'll describe how to setup a
Pelican site like this.</p><p>I finally made a personal web site with <a href="http://blog.getpelican.com/">Pelican</a>
and Alexandre Vicenzi's great <a href="https://github.com/alexandrevicenzi/Flex/">Flex</a>
theme :)!</p>
<p>Stay tuned for a follow-up blog post, where I'll describe how to setup a
Pelican site like this.</p>