Continued corespotlightd process CPU overload issues

Just to increment the anecdata.

I have this problem, new MBA (M3), 1TB drive, 16GB rams.

In frustration, I both rage quit pages and deleted metadata/corespotlight folder. Problem went away. I had half dozen pages files open, all <3MB, nothing but text. I launched pages with one file, 6000 word outline. Everything awesome for about 14 hours. Then corespotlight up and running nonstop for like 4-5 hours. Rage-quit pages again. A flurry of activity, then cpu died down, corespotlightd not in the top 20 in activity monitor.

Sent feedback via assistant to Apple, reported here. Problem does seem pages related, which is sad, because I was growing very familiar with and fond of pages over Word. Apple! Please Fix!

rage-quit means right clicking dock icon->quit while cursing loudly.

Mitch Stone wrote:

The inconsistency of this problem from day to day and Mac to Mac is crazy. I am presently not experiencing it on my M2 Ultra Studio. I never experienced it on my M1 MBA, even when opening the same Pages file on it as on the Studio. All this said, I am not convinced that preference files are not implicated, because the OS writes to some them, including the spotlight plist file. I suppose if this bug was a simple one Apple would have fixed it already.

The most bizarre thing that has happened to me so far in this whole misadventure was that before I determined that it was permissible to delete CoreSpotlight metadata without e.g. ending up with an un-bootable system, I was contending with multiple Time Machine issues, kernel panics, temporary system lockups (especially during video playback), and corespotlightd pinning the CPUs on my M1 Ultra Mac Studio (pinning the CPUs on a system like that is no easy task). Okay, that's what many people on this thread have reported. But here's where it gets weird:

I have never deleted Spotlight metadata on this system. That metadata has never exceeded about 40 GB, which in my months-long experience is not generally enough to precipitate these kinds of issues, at least, not on any of the Macs I own. But, after I deleted enormous quantities (> 500 GB) of Spotlight metadata on two other Macs (my iMac Pro and my 27-inch iMac), the problems I was experiencing on my Mac Studio literally disappeared, never (or at least not yet, a month later) to reappear.

I cannot account for why this would be, other than that all three systems (and my MBP) are all on the same iCloud account, and all four have iCloud sync turned on for, among other things, Pages.

Is this just a lucky coincidence? I'm leaning in that direction. But the timing is certainly suggestive of, well…something.

ericmurphysf wrote:

If anything 15.4 seems to have worsened the problem of extremely rapid buildup of Spotlight metadata.

Me too. My metadata folder had stayed between 2 and 10 Gb for months. With 15.4.1 I'm over 30Gb.

The next experiment will be to quit Pages for a while and see if metadata comes down in size.

Also bad news here, for me. Prior to 15.4 the folder would shrink quickly after I closed all Pages files. Now it's stubbornly stuck about 30Gb.

Mitch Stone wrote:

No evidence points to iCloud being implicated in this issue in any way whatsoever. Certain large Pages documents that have been extensively edited appear to be the most common trigger of this issue, and it is not dependent on them being stored locally or in the cloud.

From what I can tell, what triggers rapid accumulation of Spotlight data specifically with large Pages documents is the way macOS automatically does versioning of such documents. That versioning is not due to sync via iCloud; it's simply the design decision of allowing the user to revert to any prior version of the doc, or indeed copying data from earlier versions to the current version if e.g. you accidentally deleted a paragraph or section of a document.

My hypothesis (so far not confirmed by Apple) is that the rapid growth of Spotlight metadata is an artifact of the various daemons doing Spotlight indexing (md_worker, etc.) reindexing the entire document, rather than just newly-edited sections. This happens whether or not you have iCloud sync turned on.

As long as I do not use Pages, all is fine with the CPU usage. There is some problem between Pages and OS that is doing this.

I am using an M4 iMac. I've been posting about these problems for some time in this and other threads. I wanted to update what I've already posted.

I am still seeing all the documented issues with corespotlight folder sizes increasing and regularly delete folders in ~/Library/Metadata/CoreSpotlight/ when they exceed 4 GB in Tahoe. This solves the issues.

Curiously, I left for a long appointment this morning with these folders as large as 14 GB but didn't have time to delete them as usual. When I returned several hours later, the folder sizes had dropped back to around 1 GB.

I primarily use Pages and first saw the issues in Pages. Lately I have been viewing large mp4 movie files with QuickTime from a webcam I leave on at night for security. I record with OBS Studio on an Intel 21.5 iMac I upgraded to an SSD from the Fusion Drive. I saw the corespotlight folders increase on the M4 iMac after watching the movies for just over an hour for night time movement, at a speed of 10X and occasionally stopping to make screenshot movie captures of nighttime activity.

I sometimes notice Safari running slowly and large folders are always the culprit. Again the issue is resolved by deleting large folders.

My M1 MBA is using Tahoe and has no problems.

I am hoping to add to the information already known about this problem.

I checked Activity Monitor on my M2 MBA 8 hours after turning off Apple Intelligence and the CPU usage for corespotlightd was 0.0% !

Very similar to my experience.

Timing-wise, it corresponds with the release of the Apple Intelligence software (that's when I started noticing some odd behaviour). I temporarily have disabled Apple Intelligence to see if it catches up and settles down with it disabled (I'm not overly optimistic about that). At the moment I'm watching corespotlightd running up to a peak of 152% CPU, and then going gradually to a low of 18%.

Like yourself, Apple Intelligence I have an M1 MBP that is behaving just fine (but Apple Intelligence is disabled on that machine because apparently it's not available with "English - Canada" selected as a language).

About 2 weeks ago, before I did a re-install of MacOS, I ran Disk Utility from within Recovery, and it spewed a ton of index count errors - don't remember the specific text, but they were all the same, and all clearly some kind of "off-by-one" counting error.

It is marginally possible that I have a failing SSD in the machine, but that seems unlikely.

Firstly, thank you deeply to the person that made this thread and came to the realization about Pages. I was going crazy trying to research corespotlightd and having nothing work. Secondly, is this likely to be fixed with Sequoia 15.3? Finally, while I don't have much to add other than confirmations of what's already been said, I have noticed that corespotlightd will inevitably start to bug after a while no matter how small the document is. It simply takes longer. If I open my big document of hundreds of pages, it's immediately. But I've also had open files that are just 2 pages, and while everything remained calm for a while, after about 30 minutes it's back to acting up again, despite iCloud for Pages and Apple Intelligence being deactivated.

Also, before disabling any iCloud app sync I didn't need, I was also having major issues with several iCloud daemons (technically still am, just to a much lesser extent). This whole thing feels like an Apple Intelligence related screw up, where (for whatever reason) the system is wrongfully tasked with a process it cannot complete, or, it is unable to recognize the task has already been completed. I don't mean this as in Apple Intelligence the service itself, I mean regarding whatever software changes they made to the OS outside of Apple Intelligence in order to accommodate for it (if that makes sense).

I upgraded just a week ago from a base model M1 MacBook Air to an M4 Pro Macbook Pro and on that Air I didn't get to update to 15.2 because I was having issues getting the update to download properly, but I'm grateful, cause I never had any issues on that device related to Pages or iCloud. This M4 Pro came with 15.2 already installed, and I believe I had to update my iWork apps for the current version (which I regret). Whenever Pages is opened the fans will inevitably kick in (CPU increase), the battery drain is as severe as if I was using DaVinci Resolve, and the disk will write over 80mb every 10 or so seconds. Whenever I force quit corespotlightd it simply restarts extremely aggressively, writing over 100mb every second. At its absolute worst, before I found this thread and tried the remedies, I had lost over 60GB of storage space in just a couple hours (labeled as System Data). The disk write per 8-10 seconds was in the low hundreds.

Last thing worth noting, when I tried logging into my Apple ID during the initial setup for my M4 Pro (with Sequoia 15.2 already installed), I would get the rainbow wheel about 10 seconds after doing so. I would reach the screen for setting up the system and profile name, and then I'd get hit with the rainbow. This happened to me twice in a row before I decided to just skip logging into my Apple ID upon setup, and when I did that, the device setup went perfectly fine. Not sure if this is related, but considering all the iCloud issues I figured it's worth noting.

As an experiment, I left Pages running over the weekend on my work computer which was otherwise idle all weekend. The only Pages file open was a single document that's all of 145 kB. Nevertheless, the Spotlight metadata folders on this system have exploded since Friday evening, going from 63 GB to 123 GB. Otherwise the system seems to be running normally, with the CPUs at about 85% idle and corespotlightd nowhere to be seen in Activity Monitor (it's using about 3% CPU).

Nevertheless, having learned my lesson, the first thing I'll be doing when I get in to work tomorrow is simply deleting the metadata folders. I've done this three times before with no ill effects, it's completely resolved the Spotlight and Time Machine issues I've been having, and until Apple resolves this issues (assuming they ever do), it's a quick, easy, and relatively painless solution. Having to delete a couple of folders every few weeks doesn't seem that onerous, even if I have to do it indefinitely.

Have tried deleting part of metadata - the contents of NSFFileProtectionCompleteUntilFirstAuthentication and Priority.

Since then, used Pages without obvious problems. And I purposely did lots and lots of edits all over. The Metadata did grow a bit - but then shrank when I closed Pages.

On my MBP, same folders are also larger than I would have expected (a bit over 20GB each of the above). But I've had hardly any issues on the MBP - had loads on the M4 mini.

But when I looked inside I found a very large number of files named journalAttr.<number>_toc.

And that just might correspond to my Pages issues. The documents which have always seemed to be the most sensitive to issues have significant and dense Tables of Contents.

I will keep looking.

I know I was on this thread some time ago then dropped it. Afraid personal issues got in the way of trying to get it resolved.

I have iMac 27" intel. I delete System Data related to Library>Metadata>"CoreSpotlight" and "Spotlight KnowledgeEvents" EVERY NIGHT (and delete from Trash, too), or I hit the 50GB threshold that affects pace of storage accumulation and performance significantly. Thanks to this Group for this bandage to keep operational.

My tech knowledge is maxed out at starting my lawnmower. Amazed that I found this group. Apple Support let me down badly. At year end, I began to get pop-ups about "Your system has run out of application memory." Apple support baffled. Ultimately they had me delete and refill MacOS and all data, which is still causing me grief. All for nothing. Sigh...