Verschillen

Dit geeft de verschillen weer tussen de geselecteerde revisie en de huidige revisie van de pagina.

--- werkinstructies:herstart_filebeat_vanaf_eerdere_dag [2017/02/10 10:31]
abel [Filebeat Data overdracht herstarten]
+++ werkinstructies:herstart_filebeat_vanaf_eerdere_dag [2017/02/10 12:06] (huidige)
abel [Elasticsearch cleanup]
@@ Regel 1: / Regel 1: @@
-====== Filebeat Data overdracht herstarten ======
+====== Filebeat Data overdracht herstarten vanaf een eerder moment ======
 Er zijn situaties waar een misconfiguratie in logstash, het per ongeluk uitstaan van logstash of een corruptie in de elasticsearch data het nodig maken om log data, vanaf een bepaalde dag, opnieuw naar de ELK server te krijgen.
 ===== Voorbereiding =====
-stop filebeat op de betreffende clients. log in alle betreffende nodes en doe:
+stop filebeat op de betreffende clients. log in alle betreffende nodes en doe:\\
 ''service filebeat stop''
-stop logstash op de ELK server. log in op de ELK server en doe:
+stop logstash op de ELK server. log in op de ELK server en doe:\\
 ''service logstash stop''
@@ Regel 17: / Regel 13: @@
 ===== Elasticsearch cleanup =====
-Als er een parsefout is opgetreden of als er partial data is moeten de betreffende indices verwijderd worden in elasticsearch.
+Als er een parsefout is opgetreden of als er partial data is moeten de betreffende indices verwijderd worden in Elasticsearch.
+log in op de Elasticsearch node en  vindt de index , list all indices: ''%%curl 'localhost:9200/_cat/indices?v' | grep filebeat%%''
+verwijder de index door: ''%%curl -XDELETE 'localhost:9200/filebeat-index-naam'%%''
+===== filebeat cliënt datum reset =====
+De filebeat cliënt houdt in een registry bij welke files er al ge-parced zijn. Door de entry van de laatste dagen weg te gooien laat je filebeat geloven dat die nog niet verwerkt zijn.
+''vi /var/lib/filebeat/registry''
+Helaas zitten er geen line returns in deze file dus is de file wat lastig leesbaar:
+<code>
+(https://www.dokuwiki.org/plugin:wrap nog toevoegen aan dociwiki)
+{"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-11.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-11.log","offset":3192204,"FileStateOS":{"inode":5376524,"device":64
+}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-12.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-12.log","offset":2440847,"FileStateOS":{"inode":5376525,"devic
+e":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-13.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-13.log","offset":2687084,"FileStateOS":{"inode":5376357,"
+device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-14.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-14.log","offset":2430752,"FileStateOS":{"inode":5376
+,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-15.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-15.log","offset":2474377,"FileStateOS":{"inode"
+:5376400,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-16.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-16.log","offset":2417372,"FileStateOS":{"i
+node":5376407,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-17.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-17.log","offset":2722408,"FileStateOS
+":{"inode":5376392,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-18.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-18.log","offset":3141147,"FileSt
+ateOS":{"inode":5376428,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-19.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-19.log","offset":2823749,"F
+ileStateOS":{"inode":5376361,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-20.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-20.log","offset":35391
+,"FileStateOS":{"inode":5376430,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-21.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-21.log","offset":
+2982598,"FileStateOS":{"inode":5376462,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-22.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-22.log","off
+set":2624898,"FileStateOS":{"inode":5376528,"device":64771}},"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-23.log":{"source":"/opt/think/log/TA/node01/jboss/localhost_access_log.2016-12-23.log"
+,"offset":3028418,"FileStateOS":{"inode":5376492,"device":64771}}}
+</code>
+Maar de basis opmaak is dus:
+<code>
+{   "/var/log/localhost_access_log.2016-10-14.log":
+       {"source":"/var/log/localhost_access_log.2016-10-14.log","offset":2750260,"FileStateOS":
+          {"inode":3410337,"device":64771}}
+    ,
+    "/var/log/localhost_access_log.2016-10-15.log":
+       {"source":"/var/log/localhost_access_log.2016-10-15.log","offset":2960699,"FileStateOS":
+          {"inode":3410511,"device":64771}}
+    ,
+    "/var/log/localhost_access_log.2017-02-10.log":
+       {"source":"/var/log/localhost_access_log.2017-02-10.log","offset":308034,"FileStateOS":
+          {"inode":3410407,"device":64771}}
+}
+</code>
+Dus de laatste accolade laten staan en de voorgaande comma ook verwijderen.
-log in op de elasticsearch node en  vindt de index , list all indices:
+Vervolgens is het belangrijk om te checken in de file ''/etc/filebeat/filebeat.yml'' of de waarden voor
+'' ignore_older: 51h'' en ''close_older: 50h'' wijd genoeg staan om de files opnieuw in te lezen. Overigens moet ignore_older groter zijn dan close_older.
-''%%curl 'localhost:9200/_cat/indices?v%%''
-verwijder de index door:
+===== afronden =====
-''%%curl -XDELETE 'localhost:9200/indexnaam'%%''
+Start logstash op de ELK server: ''service logstash start''
+en start filebeat op de betreffende clients: ''service filebeat start''

Gebruikershulpmiddelen

Site-hulpmiddelen

Verschillen

Paginahulpmiddelen