– Looking at the crawl DB log in SharePoint, the incremental crawl was running every 1.5 hours with 0 success and 100% failures.
– There were about 961K errors with each incremental crawl scheduled at every 2 hours.
– There is a temporary folder that Crawl uses to place files temporarily on the server. The path is C:\Users\”SPAdminAccount”\AppData\Local\Temp\gthrsvc_OSearch14\
– I found out that this temporary folder was somehow missing / deleted.
On creating the above mentioned folder manually, the success rate of crawl went high and there were very less failures (as expected)
Post this, the space utilization went about 10 MB a day which is expected during crawl.