这是一个创建于 984 天前的主题，其中的信息可能已经有所发展或是发生改变。
由于我的网站访问人数越来越多，nginx 的日志大小也在指数级增长，想问一下大家有没有 nginx 日志分析的工具，即我将 nginx 日志复制出来，然后通过该工具进行数据的相关分析（错误类型、各地区 IP 访问量等）
 | 1 zerofiny 2023-05-02 10:08:57 +08:00 goaccess |
 | 2 julyclyde 2023-05-02 10:36:23 +08:00 如果真的是指数级那估计是被攻击了 应该线性增长才对啊 |
 | 3 seers 2023-05-02 10:43:30 +08:00 一般都是用 prometheus |
 |
 | 5 atonst 2023-05-02 10:46:23 +08:00 logstalgia |
 | 6 LxnChan OP |
 | 7 PolarBears 2023-05-02 11:50:24 +08:00 默认格式就 goaccess 吧,如果有条件的话还是建议配置 nginx 日志格式为 json 格式然后丢 elasticsearch 上分析吧 |
 | 8 tiga99 2023-05-02 17:32:37 +08:00 如果资源足够，可以将日志写到 es ，用 grafana 做展示；参考博客： https://www.xiaoleizhang.com/index.php/archives/120/ 如果资源不足或者不想这么复杂，可以将 nginx 重新编译一下，加上 nginx-module-vts 和 geoip2 模块，大概效果如下： ```text # HELP nginx_vts_filter_bytes_total The request/response bytes # TYPE nginx_vts_filter_bytes_total counter # HELP nginx_vts_filter_requests_total The requests counter # TYPE nginx_vts_filter_requests_total counter # HELP nginx_vts_filter_request_seconds_total The request processing time in seconds counter # TYPE nginx_vts_filter_request_seconds_total counter # HELP nginx_vts_filter_request_seconds The average of request processing times in seconds # TYPE nginx_vts_filter_request_seconds gauge # HELP nginx_vts_filter_request_duration_seconds The histogram of request processing time # TYPE nginx_vts_filter_request_duration_seconds histogram # HELP nginx_vts_filter_cache_total The requests cache counter # TYPE nginx_vts_filter_cache_total counter nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="unknow",direction="in"} 7332314 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="unknow",direction="out"} 504487933 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="unknow",code="1xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="unknow",code="2xx"} 25577 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="unknow",code="3xx"} 6 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="unknow",code="4xx"} 4 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="unknow",code="5xx"} 0 nginx_vts_filter_request_seconds_total{filter="a.example.com",filter_name="unknow"} 0.000 nginx_vts_filter_request_seconds{filter="a.example.com",filter_name="unknow"} 0.000 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="miss"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="bypass"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="expired"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="stale"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="updating"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="revalidated"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="hit"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="unknow",status="scarce"} 0 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="四川省",direction="in"} 3647380 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="四川省",direction="out"} 5444493 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="四川省",code="1xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="四川省",code="2xx"} 12939 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="四川省",code="3xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="四川省",code="4xx"} 6441 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="四川省",code="5xx"} 0 nginx_vts_filter_request_seconds_total{filter="a.example.com",filter_name="四川省"} 0.000 nginx_vts_filter_request_seconds{filter="a.example.com",filter_name="四川省"} 0.000 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="0.100"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="0.200"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="0.500"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="1.000"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="3.000"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="5.000"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="10.000"} 19380 nginx_vts_filter_request_duration_seconds_bucket{filter="a.example.com",filter_name="四川省",le="+Inf"} 19380 nginx_vts_filter_request_duration_seconds_sum{filter="a.example.com",filter_name="四川省"} 0.000 nginx_vts_filter_request_duration_seconds_count{filter="a.example.com",filter_name="四川省"} 19380 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="miss"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="bypass"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="expired"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="stale"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="updating"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="revalidated"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="hit"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="四川省",status="scarce"} 0 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="overseas",direction="in"} 2160 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="overseas",direction="out"} 4680 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="overseas",code="1xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="overseas",code="2xx"} 18 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="overseas",code="3xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="overseas",code="4xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="overseas",code="5xx"} 0 nginx_vts_filter_request_seconds_total{filter="a.example.com",filter_name="overseas"} 0.000 nginx_vts_filter_request_seconds{filter="a.example.com",filter_name="overseas"} 0.000 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="miss"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="bypass"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="expired"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="stale"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="updating"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="revalidated"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="hit"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="overseas",status="scarce"} 0 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="江西",direction="in"} 14156529 nginx_vts_filter_bytes_total{filter="a.example.com",filter_name="江西",direction="out"} 19690820 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="江西",code="1xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="江西",code="2xx"} 74493 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="江西",code="3xx"} 0 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="江西",code="4xx"} 1000 nginx_vts_filter_requests_total{filter="a.example.com",filter_name="江西",code="5xx"} 0 nginx_vts_filter_request_seconds_total{filter="a.example.com",filter_name="江西"} 0.000 nginx_vts_filter_request_seconds{filter="a.example.com",filter_name="江西"} 0.000 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="miss"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="bypass"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="expired"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="stale"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="updating"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="revalidated"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="hit"} 0 nginx_vts_filter_cache_total{filter="a.example.com",filter_name="江西",status="scarce"} 0 ``` |
 | 9 davidyin 2023-05-02 18:25:24 +08:00 via Android Awstat |
 | 10 eroko 2023-05-02 19:21:10 +08:00 我们用的是 loki |
 | 13 changdig 2023-05-03 22:04:13 +08:00 我是简单写了个 py 脚本去处理 nginx 日志然后写库或者写 csv 文件 |
 | 14 killva4624 2023-05-04 10:50:38 +08:00 elk 或者 loki ，把日志结构化入库分析。 |
Select Language