Skip to content

HOWTO: Install and configure Graphite on FreeBSD 9.2 and simple benchmark

October 8, 2013

This document continues and can be linked from:

1. Create the FreeBSD server with zfs on root:

HOWTO: FreeBSD ZFS on root partition (zroot) (freebsd-zfs) MUST-BE-EDIT***

2.  Adding SSD as a ZFS write cache and addin SSD as a ZFS read cache:

HOWTO: ZFS add SSD as write (L2ARC) cache .. and performance statistics

HOWTO: FreeBSD ZFS add SSD as log/ZIL read cache to ROOT POOL

So if you install the server from the bare metal , you can use this howtos to prepare is for the particular HOWTO: Install Graphite on FreeBSD 9.2

So, lets proceed

0. Like ALWAYS you have to carefully read the graphite docs: http://graphite.wikidot.com/installation

1. First install git

# cd /usr/ports/devel/git && make config 

# make install clean clean-depends

………………………………………

here it will install and make config on a lot of packages, be careful in your choices 🙂

2. Install pycairo

# cd /usr/ports/graphics/py-cairo ( I use python 2.7 so I will use py-cairo for python 2 bindings )

# make config

# make install clean clean-depends

………………………………………

===> Registering installation for py27-cairo-1.10.0_1
===> Cleaning for cairo-1.10.2_6,2
===> Cleaning for xcb-util-renderutil-0.3.8
===> Cleaning for pixman-0.30.2
===> Cleaning for freetype2-2.5.0.1
===> Cleaning for png-1.5.17
===> Cleaning for fontconfig-2.10.95,1
===> Cleaning for glib-2.36.3
===> Cleaning for pcre-8.33
===> Cleaning for xorg-macros-1.17.1
===> Cleaning for libxcb-1.9.1
===> Cleaning for xcb-util-0.3.9_1,1
===> Cleaning for cmake-2.8.11.2
===> Cleaning for libffi-3.0.13
===> Cleaning for libcheck-0.9.10
===> Cleaning for xcb-proto-1.8
===> Cleaning for libpthread-stubs-0.3_3
===> Cleaning for libXau-1.0.8
===> Cleaning for libXdmcp-1.1.1
===> Cleaning for cmake-modules-2.8.11.2_1
===> Cleaning for xproto-7.0.24
===> Cleaning for py27-cairo-1.10.0_1

3. Install mod_python for apache 22 web server:

# cd /usr/ports/www/mod_python3/ && make install clean clean-depends

TADAN! PROBLEMS:

*** Warning: Linking the shared library mod_python.la against the
*** static library /usr/local/lib/python2.7/config/libpython2.7.a is not portable!
/usr/bin/ld: /usr/local/lib/python2.7/config/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `a local symbol’ can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/python2.7/config/libpython2.7.a: could not read symbols: Bad value
apxs:Error: Command failed with rc=65536
.
*** [mod_python.so] Error code 1
1 error
*** [do_dso] Error code 2
1 error
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** [do-build] Error code 1

make MAKE_JOBS_UNSAFE=yes is NOT A SOLUTION!!!, let me show you:

# make  MAKE_JOBS_UNSAFE=yes

*** Warning: Linking the shared library mod_python.la against the
*** static library /usr/local/lib/python2.7/config/libpython2.7.a is not portable!
/usr/bin/ld: /usr/local/lib/python2.7/config/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `a local symbol’ can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/python2.7/config/libpython2.7.a: could not read symbols: Bad value
apxs:Error: Command failed with rc=65536
.
*** [mod_python.so] Error code 1

Stop in /usr/ports/www/mod_python3/work/mod_python-3.3.1/src.
*** [do_dso] Error code 1

Stop in /usr/ports/www/mod_python3/work/mod_python-3.3.1.
*** [do-build] Error code 1

Stop in /usr/ports/www/mod_python3.

AS THE DJANGO DOCS SAYS: use mod_wsgi NOT MOD_PYTHON !!!!

# cd /usr/ports/www/mod_wsgi && make install

NOW, let’s proceed to install django

First we will install py-pip and will use it for django installation

# cd /usr/ports/devel/py-pip

# make config && make install clean clean-depends

Installed /usr/local/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg
===> Registering installation for py27-pip-1.4.1
===> Cleaning for py27-distribute-0.6.35
===> Cleaning for py27-pip-1.4.1

# pip install django

Downloading/unpacking django

Downloading Django-1.5.4.tar.gz (8.1MB): 8.1MB downloaded

Running setup.py egg_info for package django

warning: no previously-included files matching ‘__pycache__’ found under directory ‘*’

warning: no previously-included files matching ‘*.py[co]’ found under directory ‘*’

Installing collected packages: django

Running setup.py install for django

changing mode of build/scripts-2.7/django-admin.py from 644 to 755

warning: no previously-included files matching ‘__pycache__’ found under directory ‘*’

warning: no previously-included files matching ‘*.py[co]’ found under directory ‘*’

changing mode of /usr/local/bin/django-admin.py to 755

Successfully installed django

Cleaning up…

Test Django installation:

# python

import django

…..

Install Graphite from github ( latest stable version )

# cd /root/temp && git clone https://github.com/graphite-project/graphite-web.git

Cloning into ‘graphite-web’…
remote: Counting objects: 13921, done.
remote: Compressing objects: 100% (5423/5423), done.
remote: Total 13921 (delta 9025), reused 13058 (delta 8315)
Receiving objects: 100% (13921/13921), 17.14 MiB | 1.04 MiB/s, done.
Resolving deltas: 100% (9025/9025), done.
Checking connectivity… done

# cd graphite-web/
# git checkout 0.9.x

# cd .. && git clone https://github.com/graphite-project/carbon.git

Cloning into ‘carbon’…
remote: Counting objects: 3094, done.
remote: Compressing objects: 100% (1728/1728), done.
remote: Total 3094 (delta 1478), reused 2796 (delta 1213)
Receiving objects: 100% (3094/3094), 873.58 KiB | 399.00 KiB/s, done.
Resolving deltas: 100% (1478/1478), done.
Checking connectivity… done

cd carbon/ 

# git checkout 0.9.x

# cd .. && git clone https://github.com/graphite-project/whisper.git

Cloning into ‘whisper’…
remote: Counting objects: 494, done.
remote: Compressing objects: 100% (316/316), done.
remote: Total 494 (delta 259), reused 392 (delta 172)
Receiving objects: 100% (494/494), 96.20 KiB | 0 bytes/s, done.
Resolving deltas: 100% (259/259), done.
Checking connectivity… done

# cd whisper && git checkout 0.9.x && cd ..

Now, we proceed with the whisper, carbon and graphite-web installation and configuration:

# pushd whisper

# python setup.py install

# popd

# pushd carbon && python setup.py install && popd

Install Graphite-web

first , check for deps and install any if there are

# cd /root/temp && pushd graphite-web && python check-dependencies.py

# popd && pushd graphite-web

# python setup.py install

and Finaly, Configure apache:

# httpdexample=`find / -name ‘example-graphite-vhost.conf’` && cp $httpdexample to $apache_conf_dir/vhosts/

take out this line from virtualhost section

WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}

and restart apache.

Install some more shits:

/usr/ports/devel/python-twisted

and

/usr/ports/databases/py-sqlite3  ( cuz I miss it )

The Shits:

[Tue Oct 08 19:15:31 2013] [error] mod_wsgi (pid=1436): Target WSGI script ‘/opt/graphite/conf/graphite.wsgi’ cannot be loaded as Python module.
[Tue Oct 08 19:15:31 2013] [error] mod_wsgi (pid=1436): Exception occurred processing WSGI script ‘/opt/graphite/conf/graphite.wsgi’.

FIX:

# pip install –upgrade graphite-web==0.9.11

and # /usr/local/etc/rc.d/apache22 restart

And, little , simple stupid benchkarm

1. generate file to pipe to relay

# while /usr/bin/true; do echo srv.$RANDOM.$RANDOM `jot -r 1 20000 25000` `jot -r 1 1381400261 1381403661` >> metric-benchmark-graphite
; done

# head -100000 metric-benchmark-graphite > metric-benchmark-graphite.100k

2. pipe to carbon-relay (one relay with two carbon caches)

# date; cat metric-benchmark-graphite-100k | nc localhost 2013;date

Thu Oct 10 17:31:17 EEST 2013
Thu Oct 10 17:32:32 EEST 2013

Advertisements

From → FreeBSD, Monitoring, ZFS

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: