搭建FastDFS分布式集群通过Nginx+Lua+GraphicsMagick实现

一、需求

?根据用户的请求参数对图片动态裁剪并自动加上水印，将裁剪后的图片和原图保存在同一级目录

例如原图地址的路径为：/images/uploads/2021/09/01.jpg

按200x200裁剪后的图片路径为：/images/uploads/2021/09/01_200x200.jpg

当用户第一次请求200x200的图片时，会自动将图片动态裁剪为200x200的大小，并自动加上水印。?

二、实现方式

1、nginx自带的http_image_filter_module模块

? image filter module使用的是GD，性能、效率、处理后的图片质量不如 GraphicsMagick，并且裁剪后也不会保存，这样每次请求过来都要重新裁剪，会导致访问速度很慢。

image filter module 不会真正生成裁剪/缩放后的图片，而是通过 Nginx 直接输出的，这样每次请求或缓存过期后都需要重新裁剪/缩放，这样无疑会增加 Nginx 的负担。?

2、搭配Lua+GraphicsMagick实现

使用Lua+GraphicsMagick裁剪的图片会保存在磁盘上，用户以后再访问就不再做裁剪，而是直接取之前保存的裁剪后的图片，从而访问效率相比前者好很多。

三、FastDFS集群模块架构

?FastDFS 系统有三个角色：跟踪服务器(Tracker Server)、存储服务器(Storage Server)和客户端(Client)。

• Tracker Server: 跟踪服务器，主要做调度工作，起到均衡的作用；负责管理所有的storage server和group，每个storage在启动后会连接 Tracker，告知自己所属 group 等信息，并保持周期性心跳。多个Tracker之间是对等关系，不存在单点故障。
• Storage Server: 存储服务器，主要提供容量和备份服务；以 group 为单位，每个 group 内可以有多台 storage server，组内的storage server上的数据互为备份。
• Client: 客户端，上传下载数据的服务器。?

四、系统环境

节点

IP地址

操作系统

安装软件

Tracker Server

192.168.5.106

Centos 7.6 64位

Fastdfs

Tracker Server

192.168.5.154

Centos 7.6 64位

Fastdfs

Tracker Server

192.168.5.177

Centos 7.6 64位

Fastdfs

Storage Server

192.168.5.115

Centos 7.6 64位

Fastdfs+Nginx+Lua+GraphicsMagick

五、安装部署

全局操作（所有机器执行）

wget https://github.com/happyfish100/libfastcommon/archive/V1.0.38.tar.gzwget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gzmkdir /data/fastdfs

安装libfastcommon

tar zxvf V1.0.38.tar.gzcd libfastcommon-1.0.38/./make.sh./make.sh install

安装fastdfs

tar zxvf V5.11.tar.gzcd fastdfs-5.11/./make.sh./make.sh install

tracker节点操作

cp /etc/fdfs/client.conf.sample /etc/fdfs/client.confcp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf

storage节点操作

cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.confcp /root/fastdfs-5.11/conf/http.conf /etc/fdfs/cp /root/fastdfs-5.11/conf/mime.types /etc/fdfs/

修改tracker server配置

port=22122base_path=/data/fastdfs

启动tracker server

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf startfirewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.5.0/24" port port="22122" protocol="tcp" accept'firewall-cmd --reload

修改storage server配置

port=23000base_path=/data/fastdfsstore_path0=/data/fastdfstracker_server=192.168.5.106:22122tracker_server=192.168.5.154:22122tracker_server=192.168.5.177:22122http.server_port=8088

启动storage server

firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.5.0/24" port port="23000" protocol="tcp" accept'firewall-cmd --reload/usr/bin/fdfs_storaged /etc/fdfs/storage.conf startfdfs_monitor /etc/fdfs/storage.conf list # 查看集群状态

修改Client配置

base_path=/data/fastdfstracker_server=192.168.5.106:22122tracker_server=192.168.5.154:22122tracker_server=192.168.5.177:22122

下载相关插件

wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gztar zxvf V1.20.tar.gzcd /opt/fastdfs-nginx-module-1.20/src/cp mod_fastdfs.conf /etc/fdfs/cd /opt/git clone https://github.com/alibaba/nginx-http-concat.gitgit clone https://github.com/simpl/ngx_devel_kit.gitgit clone https://github.com/openresty/echo-nginx-module.gitwget https://github.com/openresty/lua-nginx-module/archive/v0.10.14.tar.gztar zxvf v0.10.14.tar.gzwget http://luajit.org/download/LuaJIT-2.0.4.tar.gzwget http://www.lua.org/ftp/lua-5.3.1.tar.gzwget ftp://ftp.graphicsmagick.org/pub/GraphicsMagick/1.3/GraphicsMagick-1.3.18.tar.gzyum -y install readline-devel

安装LuaJIT

tar -zxf LuaJIT-2.0.4.tar.gzcd LuaJIT-2.0.4/make && make installexport LUAJIT_LIB=/usr/local/libexport LUAJIT_INC=/usr/local/include/luajit-2.0ln -s /usr/local/lib/libluajit-5.1.so.2 /lib64/libluajit-5.1.so.2

安装lua

tar -zxvpf lua-5.3.1.tar.gzcd lua-5.3.1/make linuxmake install

安装GraphicsMagick

tar zxvf GraphicsMagick-1.3.18.tar.gzcd GraphicsMagick-1.3.18/./configure --prefix=/usr/local/GraphicsMagick --enable-sharedmake && make install

编译nginx

cd nginx-1.16.1/./configure --prefix=/data/nginx --with-http_realip_module --with-http_sub_module --with-http_flv_module --with-http_dav_module --with-http_addition_module --with-http_stub_status_module --with-openssl=/usr/local/openssl --add-module=/opt/ngx_cache_purge-2.3 --with-http_gzip_static_module --with-http_ssl_module --with-stream --with-stream_ssl_module --add-module=/opt/nginx_upstream_check_module-master --add-module=/opt/fastdfs-nginx-module-1.20/src/ --with-http_image_filter_module --with-pcre --add-module=/opt/nginx-http-concat --add-module=/opt/lua-nginx-module-0.10.14 --add-module=/opt/ngx_devel_kit --add-module=/opt/echo-nginx-module --with-ld-opt=-Wl,-rpath,$LUAJIT_LIBmakecp objs/nginx /data/nginx/sbin/

配置lua脚本

mkdir /data/nginx/conf/luacd /data/nginx/conf/lua/chmod +x fastdfs.lua

fastdfs.lua内容

-- 写入文件local function writefile(filename, info) local wfile=io.open(filename, "w") --写入文件(w覆盖) assert(wfile) --打开时验证是否出错 wfile:write(info) --写入传入的内容 wfile:close() --调用结束后记得关闭end -- 检测路径是否目录local function is_dir(sPath) if type(sPath) ~= "string" then return false end local response = os.execute( "cd " .. sPath ) if response == 0 then return true end return falseend -- 检测文件是否存在local file_exists = function(name) local f=io.open(name,"r") if f~=nil then io.close(f) return true else return false endend -- 反向查找路径function last_find(str, k) local ts = string.reverse(str); local _, i = string.find(ts, k); return string.len(ts) - i + 1;end local area = nillocal originalUri = ngx.var.uri;local originalFile = ngx.var.file;local index = last_find(ngx.var.uri, "([0-9]+)x([0-9]+)"); if index then originalUri = string.sub(ngx.var.uri, 0, index-2); area = string.sub(ngx.var.uri, index); index = string.find(area, "([.])"); area = string.sub(area, 0, index-1); local index = last_find(originalFile, "([0-9]+)x([0-9]+)"); originalFile = string.sub(originalFile, 0, index-2)end -- check original fileif not file_exists(originalFile) then local fileid = string.sub(originalUri, 2); -- main local fastdfs = require('restyfastdfs') local fdfs = fastdfs:new() fdfs:set_tracker("0.0.0.0", 22122) fdfs:set_timeout(1000) fdfs:set_tracker_keepalive(0, 100) fdfs:set_storage_keepalive(0, 100) local data = fdfs:do_download(fileid) if data then -- check image dir if not is_dir(ngx.var.image_dir) then os.execute("mkdir -p " .. ngx.var.image_dir) end writefile(originalFile, data) endend -- 创建缩略图local image_sizes = {"800x800","710x300","735x250","250x150","186x150","122x122","120x120","345x345","295x295","292x292","262x262","274x274","190x190","150x150","144x144","110x110","690x340","72x72","100x100","180x180","480x240","750x740","216x216","490x190","126x126"};function table.contains(table, element) for _, value in pairs(table) do if value == element then return true end end return false end if table.contains(image_sizes, area) then local bg; if string.lower(string.sub(ngx.var.file,-3))=="jpg" then bg=" -background white "; else bg=" -background transparent "; end; local command1 = "/usr/local/GraphicsMagick/bin/gm convert -quality 90 " .. originalFile .. " -thumbnail " .. area .. bg .. " -gravity center -extent " .. area .. " " .. ngx.var.file; local command2 =""; if area == "800x800" then command2 = "/usr/local/GraphicsMagick/bin/gm composite -geometry +100+100 -dissolve 50 /data/nginx/image/yaotu1.png " .. ngx.var.file .. " " ..ngx.var.file; else command2 = "/usr/local/GraphicsMagick/bin/gm composite -geometry +60+60 -dissolve 50 /data/nginx/image/yaotu2.png " .. ngx.var.file .. " " ..ngx.var.file; end; os.execute(command1); os.execute(command2); end; if file_exists(ngx.var.file) then --ngx.req.set_uri(ngx.var.uri, true); ngx.exec(ngx.var.uri)else ngx.exit(404)end

上传水印图片

mkdir /data/nginx/image/

添加nginx配置

server { listen 8088; server_name localhost; location /hello { default_type 'text/plain'; content_by_lua 'ngx.say("hello,lua")'; } location ~/group[0-9]/M00 { #root /data/fastdfs; alias /data/fastdfs/data; set $image_root "/data/fastdfs/data"; if ($uri ~ "/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/(.*)") { set $image_dir "$image_root/$3/$4/"; set $image_name "$5"; set $file "$image_dir$image_name"; } if (!-f $file) { content_by_lua_file "/data/nginx/conf/lua/fastdfs.lua"; } ngx_fastdfs_module; } access_log /data/nginx/logs/img_access.log main; }

六、测试

上传图片

fdfs_upload_file /etc/fdfs/client.conf ./c.jpg

?上传成功后图片路径为

group1/M00/00/00/wKgFc2FEJuKAX9rKAAMGDjihSg4931.jpg?

访问图片

??http://xxx.xxx.xxx.xxx:8088/group1/M00/00/00/wKgFc2FEJuKAX9rKAAMGDjihSg4931.jpg??

?访问缩略图

??http://xxx.xxx.xxx.xxx:8088/group1/M00/00/00/wKgFc2FEJuKAX9rKAAMGDjihSg4931.jpg_735x250.jpg????