图片处理环境构建

问题

对存储的文件进行并发处理，由于处理的图片量比较大。
图片处理又比较耗内存，为防止影响正常的服务，故将其与业务服务分开部署

方案

python+uwsgi+imagemagick

环境安装：

1.配置
     16Core,16G内存，50G硬盘
2.端口
    13814（fcp），8010(http)
3.安装 graphicsmagick
     jasper-1.900.1.zip  jpegsrc.v9a.tar.gz

wget http://www.imagemagick.org/download/delegates/jpegsrc.v9a.tar.gz
wget http://www.ece.uvic.ca/~frodo/jasper/software/jasper-1.900.1.zip
gm 命令ln /usr/local/graphicsmagick/bin/gm usr/bin/gm

4.装音频插件和字体
http://my.oschina.net/ethan09/blog/372435?fromerr=WsZSucMx
安装微软雅黑字体

5.python版本升级
http://blog.csdn.net/jcjc918/article/details/11022345  由于django必须要用2.7，所以这里要进行升级
注意修改完后  要修改/usr/bin/supervisord,/usr/bin/supervisorctl 的头为/usr/bin/python2.6

6.重新安装pip和c++库
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

安装c++库
sudo yum install GraphicsMagick-c++-devel
sudo yum install boost-devel
7.安装python插件

sudo pip install -r requirements.txt
requirements.txt内容
Django==1.9.7
beautifulsoup4==4.4.1
protobuf==2.6.1
pgmagick==0.6.2
uWSGI==2.0.13.1
wsgiref==0.1.2
django-log-request-id==1.0.0

8.安装nginx
yum install nginx
9.安装uwsgi
sudo yum install uwsgi
sudo yum install uwsgi-plugin-python

nginx配置

location / {
  uwsgi_pass unix:///tmp/uwsgi.sock;
  include /etc/nginx/uwsgi_params; # the uwsgi_params file you installed
 }

client_max_body_size 20M;

10.安装图片处理服务
cd /opt/fs/
git clone http://git.firstshare.cn/Qixin/FSPythonWSGIProcess.git
cd FSPythonWSGIProcess

uwsgi --ini online_uwsgi.ini --buffer-size 20971520   --daemonize ./logs/uwsgi.log

11.异常
*** Starting uWSGI 2.0.12 (64bit) on [Tue Jun 14 16:36:20 2016] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-16) on 02 January 2016 19:53:13
os: Linux-2.6.32-573.8.1.el6.x86_64 #1 SMP Tue Nov 10 18:01:38 UTC 2015
nodename: vlnx160170.fsceshi.com
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /home/wans/FSPythonWSGIProcess
writing pidfile to /tmp/FsPythonWSGIProcess.pid
detected binary path: /usr/sbin/uwsgi
your processes number limit is 1024
your memory page size is 4096 bytes
detected max file descriptor number: 60000
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 4
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 1476277 bytes (1441 KB) for 10 cores
*** Operational MODE: preforking ***
*** no app loaded. going in full dynamic mode ***

关闭nginx   ,启动uwsgi后再启动nginx

优化点：
1.调整nginx
worker_processes     4;
worker_cpu_affinity 1000 0100 0010 0001;
2.#cluster
upstream backend{
server unix:///tmp/uwsgi.sock;
server unix:///tmp/uwsgi1.sock;
server unix:///tmp/uwsgi2.sock;
server unix:///tmp/uwsgi3.sock;
}

location / {
                    include uwsgi_params;
uwsgi_pass backend;
        }
        
问题总结:
1.yum安装遇到如下类似问题:
        yum install uwsgi
        Loaded plugins: fastestmirror
        Loading mirror speeds from cached hostfile
         * base: mirrors.btte.net
         * extras: mirrors.btte.net
         * updates: mirrors.163.com
        No package uwsgi available.
        Error: Nothing to do
   解决方案,添加epl至yum的源即可:
   yum install http://mirrors.isu.net.sa/pub/fedora/fedora-epel/7/x86_64/e/epel-release-7-6.noarch.rpm
   参考文档:https://fedoraproject.org/wiki/EPEL/zh-cn