/doc/administration/logs.md

  1# Log system
  2
  3GitLab has an advanced log system where everything is logged so that you
  4can analyze your instance using various system log files. In addition to
  5system log files, GitLab Enterprise Edition comes with Audit Events.
  6Find more about them [in Audit Events
  7documentation](http://docs.gitlab.com/ee/administration/audit_events.html)
  8
  9System log files are typically plain text in a standard log file format.
 10This guide talks about how to read and use these system log files.
 11
 12## `production.log`
 13
 14This file lives in `/var/log/gitlab/gitlab-rails/production.log` for
 15Omnibus GitLab packages or in `/home/git/gitlab/log/production.log` for
 16installations from source. (When Gitlab is running in an environment
 17other than production, the corresponding logfile is shown here.)
 18
 19It contains information about all performed requests. You can see the
 20URL and type of request, IP address and what exactly parts of code were
 21involved to service this particular request. Also you can see all SQL
 22request that have been performed and how much time it took. This task is
 23more useful for GitLab contributors and developers. Use part of this log
 24file when you are going to report bug. For example:
 25
 26```
 27Started GET "/gitlabhq/yaml_db/tree/master" for 168.111.56.1 at 2015-02-12 19:34:53 +0200
 28Processing by Projects::TreeController#show as HTML
 29  Parameters: {"project_id"=>"gitlabhq/yaml_db", "id"=>"master"}
 30
 31  ... [CUT OUT]
 32
 33  Namespaces"."created_at" DESC, "namespaces"."id" DESC LIMIT 1 [["id", 26]]
 34  CACHE (0.0ms) SELECT  "members".* FROM "members"  WHERE "members"."source_type" = 'Project' AND "members"."type" IN ('ProjectMember') AND "members"."source_id" = $1 AND "members"."source_type" = $2 AND "members"."user_id" = 1  ORDER BY "members"."created_at" DESC, "members"."id" DESC LIMIT 1  [["source_id", 18], ["source_type", "Project"]]
 35  CACHE (0.0ms) SELECT  "members".* FROM "members"  WHERE "members"."source_type" = 'Project' AND "members".
 36  (1.4ms) SELECT COUNT(*) FROM "merge_requests"  WHERE "merge_requests"."target_project_id" = $1 AND ("merge_requests"."state" IN ('opened','reopened')) [["target_project_id", 18]]
 37  Rendered layouts/nav/_project.html.haml (28.0ms)
 38  Rendered layouts/_collapse_button.html.haml (0.2ms)
 39  Rendered layouts/_flash.html.haml (0.1ms)
 40  Rendered layouts/_page.html.haml (32.9ms)
 41Completed 200 OK in 166ms (Views: 117.4ms | ActiveRecord: 27.2ms)
 42```
 43
 44In this example we can see that server processed an HTTP request with URL
 45`/gitlabhq/yaml_db/tree/master` from IP 168.111.56.1 at 2015-02-12
 4619:34:53 +0200. Also we can see that request was processed by
 47`Projects::TreeController`.
 48
 49## `application.log`
 50
 51This file lives in `/var/log/gitlab/gitlab-rails/application.log` for
 52Omnibus GitLab packages or in `/home/git/gitlab/log/application.log` for
 53installations from source.
 54
 55It helps you discover events happening in your instance such as user creation,
 56project removing and so on. For example:
 57
 58```
 59October 06, 2014 11:56: User "Administrator" (admin@example.com) was created
 60October 06, 2014 11:56: Documentcloud created a new project "Documentcloud / Underscore"
 61October 06, 2014 11:56: Gitlab Org created a new project "Gitlab Org / Gitlab Ce"
 62October 07, 2014 11:25: User "Claudie Hodkiewicz" (nasir_stehr@olson.co.uk)  was removed
 63October 07, 2014 11:25: Project "project133" was removed
 64```
 65
 66## `githost.log`
 67
 68This file lives in `/var/log/gitlab/gitlab-rails/githost.log` for
 69Omnibus GitLab packages or in `/home/git/gitlab/log/githost.log` for
 70installations from source.
 71
 72GitLab has to interact with Git repositories but in some rare cases
 73something can go wrong and in this case you will know what exactly
 74happened. This log file contains all failed requests from GitLab to Git
 75repositories. In the majority of cases this file will be useful for developers
 76only. For example:
 77
 78```
 79December 03, 2014 13:20 -> ERROR -> Command failed [1]: /usr/bin/git --git-dir=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq/.git --work-tree=/Users/vsizov/gitlab-development-kit/gitlab/tmp/tests/gitlab-satellites/group184/gitlabhq merge --no-ff -mMerge branch 'feature_conflict' into 'feature' source/feature_conflict
 80
 81error: failed to push some refs to '/Users/vsizov/gitlab-development-kit/repositories/gitlabhq/gitlab_git.git'
 82```
 83
 84## `sidekiq.log`
 85
 86This file lives in `/var/log/gitlab/gitlab-rails/sidekiq.log` for
 87Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq.log` for
 88installations from source.
 89
 90GitLab uses background jobs for processing tasks which can take a long
 91time. All information about processing these jobs are written down to
 92this file. For example:
 93
 94```
 952014-06-10T07:55:20Z 2037 TID-tm504 ERROR: /opt/bitnami/apps/discourse/htdocs/vendor/bundle/ruby/1.9.1/gems/redis-3.0.7/lib/redis/client.rb:228:in `read'
 962014-06-10T18:18:26Z 14299 TID-55uqo INFO: Booting Sidekiq 3.0.0 with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
 97```
 98
 99## `gitlab-shell.log`
100
101This file lives in `/var/log/gitlab/gitlab-shell/gitlab-shell.log` for
102Omnibus GitLab packages or in `/home/git/gitlab-shell/gitlab-shell.log` for
103installations from source.
104
105GitLab shell is used by Gitlab for executing Git commands and provide
106SSH access to Git repositories. For example:
107
108```
109I, [2015-02-13T06:17:00.671315 #9291]  INFO -- : Adding project root/example.git at </var/opt/gitlab/git-data/repositories/root/dcdcdcdcd.git>.
110I, [2015-02-13T06:17:00.679433 #9291]  INFO -- : Moving existing hooks directory and symlinking global hooks directory for /var/opt/gitlab/git-data/repositories/root/example.git.
111```
112
113## `unicorn\_stderr.log`
114
115This file lives in `/var/log/gitlab/unicorn/unicorn_stderr.log` for
116Omnibus GitLab packages or in `/home/git/gitlab/log/unicorn_stderr.log` for
117installations from source.
118
119Unicorn is a high-performance forking Web server which is used for
120serving the GitLab application. You can look at this log if, for
121example, your application does not respond. This log contains all
122information about the state of unicorn processes at any given time.
123
124```
125I, [2015-02-13T06:14:46.680381 #9047]  INFO -- : Refreshing Gem list
126I, [2015-02-13T06:14:56.931002 #9047]  INFO -- : listening on addr=127.0.0.1:8080 fd=12
127I, [2015-02-13T06:14:56.931381 #9047]  INFO -- : listening on addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fd=13
128I, [2015-02-13T06:14:56.936638 #9047]  INFO -- : master process ready
129I, [2015-02-13T06:14:56.946504 #9092]  INFO -- : worker=0 spawned pid=9092
130I, [2015-02-13T06:14:56.946943 #9092]  INFO -- : worker=0 ready
131I, [2015-02-13T06:14:56.947892 #9094]  INFO -- : worker=1 spawned pid=9094
132I, [2015-02-13T06:14:56.948181 #9094]  INFO -- : worker=1 ready
133W, [2015-02-13T07:16:01.312916 #9094]  WARN -- : #<Unicorn::HttpServer:0x0000000208f618>: worker (pid: 9094) exceeds memory limit (320626688 bytes > 247066940 bytes)
134W, [2015-02-13T07:16:01.313000 #9094]  WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 9094) alive: 3621 sec (trial 1)
135I, [2015-02-13T07:16:01.530733 #9047]  INFO -- : reaped #<Process::Status: pid 9094 exit 0> worker=1
136I, [2015-02-13T07:16:01.534501 #13379]  INFO -- : worker=1 spawned pid=13379
137I, [2015-02-13T07:16:01.534848 #13379]  INFO -- : worker=1 ready
138```
139
140## `repocheck.log`
141
142This file lives in `/var/log/gitlab/gitlab-rails/repocheck.log` for
143Omnibus GitLab packages or in `/home/git/gitlab/log/repocheck.log` for
144installations from source.
145
146It logs information whenever a [repository check is run][repocheck] on a project.
147
148[repocheck]: repository_checks.md