This proposal is another improvement working together with HBASE-18083 to resolve our online issue (archive dir consumed more than 1. Written in Java, it is designed and developed by many engineers under the framework of Apache Software Foundation. 08:15:46,055 DEBUG .: Removing:hdfs://hbase/.archive/table/4e48ffc1ec089082c66e6d1b5f018fb5/M/729e8bc1430540cb9b2c147c90039cdc from archiveĪnd my solution is very simple: When hfiles and hlogs are archived, we set the modify time of files after rename. And when data abnormally accumulated to some huge volume in archive directory, the cleaning speed might not be enough. Configuring and deploying HBase Tutorial HBase is inspired by the Google big table architecture, and is fundamentally a non-relational, open source, and column-oriented distributed NoSQL. 08:15:46,055 DEBUG .ProtobufRpcEngine: Call: getFileInfo took 1ms This project's goal is the hosting of very large tables - billions of rows X millions of columns - atop clusters of commodity hardware. Nested Class Summary Field Summary Fields inherited from class .master.cleaner. Use Apache HBase when you need random, realtime read/write access to your Big Data. This Chore, every time it runs, will clear the HFiles in the hfile archive folder that are deletable for each HFile cleaner in the chain. I believe that you have some old snapshots and that's why you have so big archive directory. Apache HBase is the Hadoop database, a distributed, scalable, big data store. So the hfile may be deleted immediatly by HFileCleaner after it is moved to archives. Kylin persists all data (meta data and cube) in HBase You may want to export the data sometimes for whatever purposes (backup, migration, troubleshotting etc) This page describes the steps to do this and also there is a Java app for you to do this easily. The rename op will not change the modify time of the hfile. But, the modify time of the hdfs file is time when its writer is closed. And timeToLiveHFileCleaner uses the modify time of the hfile to determine if it should be deleted. TimeToLiveHFileCleaner is configed to '' in hbase-default.xml.
0 Comments
Leave a Reply. |