Source file src/runtime/mgc.go
1 // Copyright 2009 The Go Authors. All rights reserved. 2 // Use of this source code is governed by a BSD-style 3 // license that can be found in the LICENSE file. 4 5 // Garbage collector (GC). 6 // 7 // The GC runs concurrently with mutator threads, is type accurate (aka precise), allows multiple 8 // GC thread to run in parallel. It is a concurrent mark and sweep that uses a write barrier. It is 9 // non-generational and non-compacting. Allocation is done using size segregated per P allocation 10 // areas to minimize fragmentation while eliminating locks in the common case. 11 // 12 // The algorithm decomposes into several steps. 13 // This is a high level description of the algorithm being used. For an overview of GC a good 14 // place to start is Richard Jones' gchandbook.org. 15 // 16 // The algorithm's intellectual heritage includes Dijkstra's on-the-fly algorithm, see 17 // Edsger W. Dijkstra, Leslie Lamport, A. J. Martin, C. S. Scholten, and E. F. M. Steffens. 1978. 18 // On-the-fly garbage collection: an exercise in cooperation. Commun. ACM 21, 11 (November 1978), 19 // 966-975. 20 // For journal quality proofs that these steps are complete, correct, and terminate see 21 // Hudson, R., and Moss, J.E.B. Copying Garbage Collection without stopping the world. 22 // Concurrency and Computation: Practice and Experience 15(3-5), 2003. 23 // 24 // 1. GC performs sweep termination. 25 // 26 // a. Stop the world. This causes all Ps to reach a GC safe-point. 27 // 28 // b. Sweep any unswept spans. There will only be unswept spans if 29 // this GC cycle was forced before the expected time. 30 // 31 // 2. GC performs the mark phase. 32 // 33 // a. Prepare for the mark phase by setting gcphase to _GCmark 34 // (from _GCoff), enabling the write barrier, enabling mutator 35 // assists, and enqueueing root mark jobs. No objects may be 36 // scanned until all Ps have enabled the write barrier, which is 37 // accomplished using STW. 38 // 39 // b. Start the world. From this point, GC work is done by mark 40 // workers started by the scheduler and by assists performed as 41 // part of allocation. The write barrier shades both the 42 // overwritten pointer and the new pointer value for any pointer 43 // writes (see mbarrier.go for details). Newly allocated objects 44 // are immediately marked black. 45 // 46 // c. GC performs root marking jobs. This includes scanning all 47 // stacks, shading all globals, and shading any heap pointers in 48 // off-heap runtime data structures. Scanning a stack stops a 49 // goroutine, shades any pointers found on its stack, and then 50 // resumes the goroutine. 51 // 52 // d. GC drains the work queue of grey objects, scanning each grey 53 // object to black and shading all pointers found in the object 54 // (which in turn may add those pointers to the work queue). 55 // 56 // e. Because GC work is spread across local caches, GC uses a 57 // distributed termination algorithm to detect when there are no 58 // more root marking jobs or grey objects (see gcMarkDone). At this 59 // point, GC transitions to mark termination. 60 // 61 // 3. GC performs mark termination. 62 // 63 // a. Stop the world. 64 // 65 // b. Set gcphase to _GCmarktermination, and disable workers and 66 // assists. 67 // 68 // c. Perform housekeeping like flushing mcaches. 69 // 70 // 4. GC performs the sweep phase. 71 // 72 // a. Prepare for the sweep phase by setting gcphase to _GCoff, 73 // setting up sweep state and disabling the write barrier. 74 // 75 // b. Start the world. From this point on, newly allocated objects 76 // are white, and allocating sweeps spans before use if necessary. 77 // 78 // c. GC does concurrent sweeping in the background and in response 79 // to allocation. See description below. 80 // 81 // 5. When sufficient allocation has taken place, replay the sequence 82 // starting with 1 above. See discussion of GC rate below. 83 84 // Concurrent sweep. 85 // 86 // The sweep phase proceeds concurrently with normal program execution. 87 // The heap is swept span-by-span both lazily (when a goroutine needs another span) 88 // and concurrently in a background goroutine (this helps programs that are not CPU bound). 89 // At the end of STW mark termination all spans are marked as "needs sweeping". 90 // 91 // The background sweeper goroutine simply sweeps spans one-by-one. 92 // 93 // To avoid requesting more OS memory while there are unswept spans, when a 94 // goroutine needs another span, it first attempts to reclaim that much memory 95 // by sweeping. When a goroutine needs to allocate a new small-object span, it 96 // sweeps small-object spans for the same object size until it frees at least 97 // one object. When a goroutine needs to allocate large-object span from heap, 98 // it sweeps spans until it frees at least that many pages into heap. There is 99 // one case where this may not suffice: if a goroutine sweeps and frees two 100 // nonadjacent one-page spans to the heap, it will allocate a new two-page 101 // span, but there can still be other one-page unswept spans which could be 102 // combined into a two-page span. 103 // 104 // It's critical to ensure that no operations proceed on unswept spans (that would corrupt 105 // mark bits in GC bitmap). During GC all mcaches are flushed into the central cache, 106 // so they are empty. When a goroutine grabs a new span into mcache, it sweeps it. 107 // When a goroutine explicitly frees an object or sets a finalizer, it ensures that 108 // the span is swept (either by sweeping it, or by waiting for the concurrent sweep to finish). 109 // The finalizer goroutine is kicked off only when all spans are swept. 110 // When the next GC starts, it sweeps all not-yet-swept spans (if any). 111 112 // GC rate. 113 // Next GC is after we've allocated an extra amount of memory proportional to 114 // the amount already in use. The proportion is controlled by GOGC environment variable 115 // (100 by default). If GOGC=100 and we're using 4M, we'll GC again when we get to 8M 116 // (this mark is computed by the gcController.heapGoal method). This keeps the GC cost in 117 // linear proportion to the allocation cost. Adjusting GOGC just changes the linear constant 118 // (and also the amount of extra memory used). 119 120 // Oblets 121 // 122 // In order to prevent long pauses while scanning large objects and to 123 // improve parallelism, the garbage collector breaks up scan jobs for 124 // objects larger than maxObletBytes into "oblets" of at most 125 // maxObletBytes. When scanning encounters the beginning of a large 126 // object, it scans only the first oblet and enqueues the remaining 127 // oblets as new scan jobs. 128 129 package runtime 130 131 import ( 132 "internal/cpu" 133 "internal/goarch" 134 "internal/goexperiment" 135 "internal/runtime/atomic" 136 "internal/runtime/gc" 137 "unsafe" 138 ) 139 140 const ( 141 _DebugGC = 0 142 143 // concurrentSweep is a debug flag. Disabling this flag 144 // ensures all spans are swept while the world is stopped. 145 concurrentSweep = true 146 147 // debugScanConservative enables debug logging for stack 148 // frames that are scanned conservatively. 149 debugScanConservative = false 150 151 // sweepMinHeapDistance is a lower bound on the heap distance 152 // (in bytes) reserved for concurrent sweeping between GC 153 // cycles. 154 sweepMinHeapDistance = 1024 * 1024 155 ) 156 157 // heapObjectsCanMove always returns false in the current garbage collector. 158 // It exists for go4.org/unsafe/assume-no-moving-gc, which is an 159 // unfortunate idea that had an even more unfortunate implementation. 160 // Every time a new Go release happened, the package stopped building, 161 // and the authors had to add a new file with a new //go:build line, and 162 // then the entire ecosystem of packages with that as a dependency had to 163 // explicitly update to the new version. Many packages depend on 164 // assume-no-moving-gc transitively, through paths like 165 // inet.af/netaddr -> go4.org/intern -> assume-no-moving-gc. 166 // This was causing a significant amount of friction around each new 167 // release, so we added this bool for the package to //go:linkname 168 // instead. The bool is still unfortunate, but it's not as bad as 169 // breaking the ecosystem on every new release. 170 // 171 // If the Go garbage collector ever does move heap objects, we can set 172 // this to true to break all the programs using assume-no-moving-gc. 173 // 174 //go:linkname heapObjectsCanMove 175 func heapObjectsCanMove() bool { 176 return false 177 } 178 179 func gcinit() { 180 if unsafe.Sizeof(workbuf{}) != _WorkbufSize { 181 throw("size of Workbuf is suboptimal") 182 } 183 // No sweep on the first cycle. 184 sweep.active.state.Store(sweepDrainedMask) 185 186 // Initialize GC pacer state. 187 // Use the environment variable GOGC for the initial gcPercent value. 188 // Use the environment variable GOMEMLIMIT for the initial memoryLimit value. 189 gcController.init(readGOGC(), readGOMEMLIMIT()) 190 191 // Set up the cleanup block ptr mask. 192 for i := range cleanupBlockPtrMask { 193 cleanupBlockPtrMask[i] = 0xff 194 } 195 196 work.startSema = 1 197 work.markDoneSema = 1 198 lockInit(&work.sweepWaiters.lock, lockRankSweepWaiters) 199 lockInit(&work.assistQueue.lock, lockRankAssistQueue) 200 lockInit(&work.strongFromWeak.lock, lockRankStrongFromWeakQueue) 201 lockInit(&work.wbufSpans.lock, lockRankWbufSpans) 202 lockInit(&gcCleanups.lock, lockRankCleanupQueue) 203 } 204 205 // gcenable is called after the bulk of the runtime initialization, 206 // just before we're about to start letting user code run. 207 // It kicks off the background sweeper goroutine, the background 208 // scavenger goroutine, and enables GC. 209 func gcenable() { 210 // Kick off sweeping and scavenging. 211 c := make(chan int, 2) 212 go bgsweep(c) 213 go bgscavenge(c) 214 <-c 215 <-c 216 memstats.enablegc = true // now that runtime is initialized, GC is okay 217 } 218 219 // Garbage collector phase. 220 // Indicates to write barrier and synchronization task to perform. 221 var gcphase uint32 222 223 // The compiler knows about this variable. 224 // If you change it, you must change builtin/runtime.go, too. 225 // If you change the first four bytes, you must also change the write 226 // barrier insertion code. 227 // 228 // writeBarrier should be an internal detail, 229 // but widely used packages access it using linkname. 230 // Notable members of the hall of shame include: 231 // - github.com/bytedance/sonic 232 // 233 // Do not remove or change the type signature. 234 // See go.dev/issue/67401. 235 // 236 //go:linkname writeBarrier 237 var writeBarrier struct { 238 enabled bool // compiler emits a check of this before calling write barrier 239 pad [3]byte // compiler uses 32-bit load for "enabled" field 240 alignme uint64 // guarantee alignment so that compiler can use a 32 or 64-bit load 241 } 242 243 // gcBlackenEnabled is 1 if mutator assists and background mark 244 // workers are allowed to blacken objects. This must only be set when 245 // gcphase == _GCmark. 246 var gcBlackenEnabled uint32 247 248 const ( 249 _GCoff = iota // GC not running; sweeping in background, write barrier disabled 250 _GCmark // GC marking roots and workbufs: allocate black, write barrier ENABLED 251 _GCmarktermination // GC mark termination: allocate black, P's help GC, write barrier ENABLED 252 ) 253 254 //go:nosplit 255 func setGCPhase(x uint32) { 256 atomic.Store(&gcphase, x) 257 writeBarrier.enabled = gcphase == _GCmark || gcphase == _GCmarktermination 258 } 259 260 // gcMarkWorkerMode represents the mode that a concurrent mark worker 261 // should operate in. 262 // 263 // Concurrent marking happens through four different mechanisms. One 264 // is mutator assists, which happen in response to allocations and are 265 // not scheduled. The other three are variations in the per-P mark 266 // workers and are distinguished by gcMarkWorkerMode. 267 type gcMarkWorkerMode int 268 269 const ( 270 // gcMarkWorkerNotWorker indicates that the next scheduled G is not 271 // starting work and the mode should be ignored. 272 gcMarkWorkerNotWorker gcMarkWorkerMode = iota 273 274 // gcMarkWorkerDedicatedMode indicates that the P of a mark 275 // worker is dedicated to running that mark worker. The mark 276 // worker should run without preemption. 277 gcMarkWorkerDedicatedMode 278 279 // gcMarkWorkerFractionalMode indicates that a P is currently 280 // running the "fractional" mark worker. The fractional worker 281 // is necessary when GOMAXPROCS*gcBackgroundUtilization is not 282 // an integer and using only dedicated workers would result in 283 // utilization too far from the target of gcBackgroundUtilization. 284 // The fractional worker should run until it is preempted and 285 // will be scheduled to pick up the fractional part of 286 // GOMAXPROCS*gcBackgroundUtilization. 287 gcMarkWorkerFractionalMode 288 289 // gcMarkWorkerIdleMode indicates that a P is running the mark 290 // worker because it has nothing else to do. The idle worker 291 // should run until it is preempted and account its time 292 // against gcController.idleMarkTime. 293 gcMarkWorkerIdleMode 294 ) 295 296 // gcMarkWorkerModeStrings are the strings labels of gcMarkWorkerModes 297 // to use in execution traces. 298 var gcMarkWorkerModeStrings = [...]string{ 299 "Not worker", 300 "GC (dedicated)", 301 "GC (fractional)", 302 "GC (idle)", 303 } 304 305 // pollFractionalWorkerExit reports whether a fractional mark worker 306 // should self-preempt. It assumes it is called from the fractional 307 // worker. 308 func pollFractionalWorkerExit() bool { 309 // This should be kept in sync with the fractional worker 310 // scheduler logic in findRunnableGCWorker. 311 now := nanotime() 312 delta := now - gcController.markStartTime 313 if delta <= 0 { 314 return true 315 } 316 p := getg().m.p.ptr() 317 selfTime := p.gcFractionalMarkTime + (now - p.gcMarkWorkerStartTime) 318 // Add some slack to the utilization goal so that the 319 // fractional worker isn't behind again the instant it exits. 320 return float64(selfTime)/float64(delta) > 1.2*gcController.fractionalUtilizationGoal 321 } 322 323 var work workType 324 325 type workType struct { 326 full lfstack // lock-free list of full blocks workbuf 327 _ cpu.CacheLinePad // prevents false-sharing between full and empty 328 empty lfstack // lock-free list of empty blocks workbuf 329 _ cpu.CacheLinePad // prevents false-sharing between empty and wbufSpans 330 331 wbufSpans struct { 332 lock mutex 333 // free is a list of spans dedicated to workbufs, but 334 // that don't currently contain any workbufs. 335 free mSpanList 336 // busy is a list of all spans containing workbufs on 337 // one of the workbuf lists. 338 busy mSpanList 339 } 340 _ cpu.CacheLinePad // prevents false-sharing between wbufSpans and spanWorkMask 341 342 // spanqMask is a bitmap indicating which Ps have local work worth stealing. 343 // Set or cleared by the owning P, cleared by stealing Ps. 344 // 345 // spanqMask is like a proxy for a global queue. An important invariant is that 346 // forced flushing like gcw.dispose must set this bit on any P that has local 347 // span work. 348 spanqMask pMask 349 _ cpu.CacheLinePad // prevents false-sharing between spanqMask and everything else 350 351 // List of all spanSPMCs. 352 // 353 // Only used if goexperiment.GreenTeaGC. 354 spanSPMCs struct { 355 lock mutex // no lock rank because it's a leaf lock (see mklockrank.go). 356 all *spanSPMC 357 } 358 359 // Restore 64-bit alignment on 32-bit. 360 // _ uint32 361 362 // bytesMarked is the number of bytes marked this cycle. This 363 // includes bytes blackened in scanned objects, noscan objects 364 // that go straight to black, objects allocated as black during 365 // the cycle, and permagrey objects scanned by markroot during 366 // the concurrent scan phase. 367 // 368 // This is updated atomically during the cycle. Updates may be batched 369 // arbitrarily, since the value is only read at the end of the cycle. 370 // 371 // Because of benign races during marking, this number may not 372 // be the exact number of marked bytes, but it should be very 373 // close. 374 // 375 // Put this field here because it needs 64-bit atomic access 376 // (and thus 8-byte alignment even on 32-bit architectures). 377 bytesMarked uint64 378 379 markrootNext atomic.Uint32 // next markroot job 380 markrootJobs atomic.Uint32 // number of markroot jobs 381 382 nproc uint32 383 tstart int64 384 nwait uint32 385 386 // Number of roots of various root types. Set by gcPrepareMarkRoots. 387 // 388 // During normal GC cycle, nStackRoots == nMaybeRunnableStackRoots == len(stackRoots); 389 // during goroutine leak detection, nMaybeRunnableStackRoots is the number of stackRoots 390 // scheduled for marking. 391 // In both variants, nStackRoots == len(stackRoots). 392 nDataRoots, nBSSRoots, nSpanRoots, nStackRoots, nMaybeRunnableStackRoots int 393 394 // The following fields monitor the GC phase of the current cycle during 395 // goroutine leak detection. 396 goroutineLeak struct { 397 // Once set, it indicates that the GC will perform goroutine leak detection during 398 // the next GC cycle; it is set by goroutineLeakGC and unset during gcStart. 399 pending atomic.Bool 400 // Once set, it indicates that the GC has started a goroutine leak detection run; 401 // it is set during gcStart and unset during gcMarkTermination; 402 // 403 // Protected by STW. 404 enabled bool 405 // Once set, it indicates that the GC has performed goroutine leak detection during 406 // the current GC cycle; it is set during gcMarkDone, right after goroutine leak detection, 407 // and unset during gcMarkTermination; 408 // 409 // Protected by STW. 410 done bool 411 // The number of leaked goroutines during the last leak detection GC cycle. 412 // 413 // Write-protected by STW in findGoroutineLeaks. 414 count int 415 } 416 417 // Base indexes of each root type. Set by gcPrepareMarkRoots. 418 baseData, baseBSS, baseSpans, baseStacks, baseEnd uint32 419 420 // stackRoots is a snapshot of all of the Gs that existed before the 421 // beginning of concurrent marking. During goroutine leak detection, stackRoots 422 // is partitioned into two sets; to the left of nMaybeRunnableStackRoots are stackRoots 423 // of running / runnable goroutines and to the right of nMaybeRunnableStackRoots are 424 // stackRoots of unmarked / not runnable goroutines 425 // The stackRoots array is re-partitioned after each marking phase iteration. 426 stackRoots []*g 427 428 // Each type of GC state transition is protected by a lock. 429 // Since multiple threads can simultaneously detect the state 430 // transition condition, any thread that detects a transition 431 // condition must acquire the appropriate transition lock, 432 // re-check the transition condition and return if it no 433 // longer holds or perform the transition if it does. 434 // Likewise, any transition must invalidate the transition 435 // condition before releasing the lock. This ensures that each 436 // transition is performed by exactly one thread and threads 437 // that need the transition to happen block until it has 438 // happened. 439 // 440 // startSema protects the transition from "off" to mark or 441 // mark termination. 442 startSema uint32 443 // markDoneSema protects transitions from mark to mark termination. 444 markDoneSema uint32 445 446 bgMarkDone uint32 // cas to 1 when at a background mark completion point 447 // Background mark completion signaling 448 449 // mode is the concurrency mode of the current GC cycle. 450 mode gcMode 451 452 // userForced indicates the current GC cycle was forced by an 453 // explicit user call. 454 userForced bool 455 456 // initialHeapLive is the value of gcController.heapLive at the 457 // beginning of this GC cycle. 458 initialHeapLive uint64 459 460 // assistQueue is a queue of assists that are blocked because 461 // there was neither enough credit to steal or enough work to 462 // do. 463 assistQueue struct { 464 lock mutex 465 q gQueue 466 } 467 468 // sweepWaiters is a list of blocked goroutines to wake when 469 // we transition from mark termination to sweep. 470 sweepWaiters struct { 471 lock mutex 472 list gList 473 } 474 475 // strongFromWeak controls how the GC interacts with weak->strong 476 // pointer conversions. 477 strongFromWeak struct { 478 // block is a flag set during mark termination that prevents 479 // new weak->strong conversions from executing by blocking the 480 // goroutine and enqueuing it onto q. 481 // 482 // Mutated only by one goroutine at a time in gcMarkDone, 483 // with globally-synchronizing events like forEachP and 484 // stopTheWorld. 485 block bool 486 487 // q is a queue of goroutines that attempted to perform a 488 // weak->strong conversion during mark termination. 489 // 490 // Protected by lock. 491 lock mutex 492 q gQueue 493 } 494 495 // cycles is the number of completed GC cycles, where a GC 496 // cycle is sweep termination, mark, mark termination, and 497 // sweep. This differs from memstats.numgc, which is 498 // incremented at mark termination. 499 cycles atomic.Uint32 500 501 // Timing/utilization stats for this cycle. 502 stwprocs, maxprocs int32 503 tSweepTerm, tMark, tMarkTerm, tEnd int64 // nanotime() of phase start 504 505 // pauseNS is the total STW time this cycle, measured as the time between 506 // when stopping began (just before trying to stop Ps) and just after the 507 // world started again. 508 pauseNS int64 509 510 // debug.gctrace heap sizes for this cycle. 511 heap0, heap1, heap2 uint64 512 513 // Cumulative estimated CPU usage. 514 cpuStats 515 } 516 517 // GC runs a garbage collection and blocks the caller until the 518 // garbage collection is complete. It may also block the entire 519 // program. 520 func GC() { 521 // We consider a cycle to be: sweep termination, mark, mark 522 // termination, and sweep. This function shouldn't return 523 // until a full cycle has been completed, from beginning to 524 // end. Hence, we always want to finish up the current cycle 525 // and start a new one. That means: 526 // 527 // 1. In sweep termination, mark, or mark termination of cycle 528 // N, wait until mark termination N completes and transitions 529 // to sweep N. 530 // 531 // 2. In sweep N, help with sweep N. 532 // 533 // At this point we can begin a full cycle N+1. 534 // 535 // 3. Trigger cycle N+1 by starting sweep termination N+1. 536 // 537 // 4. Wait for mark termination N+1 to complete. 538 // 539 // 5. Help with sweep N+1 until it's done. 540 // 541 // This all has to be written to deal with the fact that the 542 // GC may move ahead on its own. For example, when we block 543 // until mark termination N, we may wake up in cycle N+2. 544 545 // Wait until the current sweep termination, mark, and mark 546 // termination complete. 547 n := work.cycles.Load() 548 gcWaitOnMark(n) 549 550 // We're now in sweep N or later. Trigger GC cycle N+1, which 551 // will first finish sweep N if necessary and then enter sweep 552 // termination N+1. 553 gcStart(gcTrigger{kind: gcTriggerCycle, n: n + 1}) 554 555 // Wait for mark termination N+1 to complete. 556 gcWaitOnMark(n + 1) 557 558 // Finish sweep N+1 before returning. We do this both to 559 // complete the cycle and because runtime.GC() is often used 560 // as part of tests and benchmarks to get the system into a 561 // relatively stable and isolated state. 562 for work.cycles.Load() == n+1 && sweepone() != ^uintptr(0) { 563 Gosched() 564 } 565 566 // Callers may assume that the heap profile reflects the 567 // just-completed cycle when this returns (historically this 568 // happened because this was a STW GC), but right now the 569 // profile still reflects mark termination N, not N+1. 570 // 571 // As soon as all of the sweep frees from cycle N+1 are done, 572 // we can go ahead and publish the heap profile. 573 // 574 // First, wait for sweeping to finish. (We know there are no 575 // more spans on the sweep queue, but we may be concurrently 576 // sweeping spans, so we have to wait.) 577 for work.cycles.Load() == n+1 && !isSweepDone() { 578 Gosched() 579 } 580 581 // Now we're really done with sweeping, so we can publish the 582 // stable heap profile. Only do this if we haven't already hit 583 // another mark termination. 584 mp := acquirem() 585 cycle := work.cycles.Load() 586 if cycle == n+1 || (gcphase == _GCmark && cycle == n+2) { 587 mProf_PostSweep() 588 } 589 releasem(mp) 590 } 591 592 // goroutineLeakGC runs a GC cycle that performs goroutine leak detection. 593 // 594 //go:linkname goroutineLeakGC runtime/pprof.runtime_goroutineLeakGC 595 func goroutineLeakGC() { 596 // Set the pending flag to true, instructing the next GC cycle to 597 // perform goroutine leak detection. 598 work.goroutineLeak.pending.Store(true) 599 600 // Spin GC cycles until the pending flag is unset. 601 // This ensures that goroutineLeakGC waits for a GC cycle that 602 // actually performs goroutine leak detection. 603 // 604 // This is needed in case multiple concurrent calls to GC 605 // are simultaneously fired by the system, wherein some 606 // of them are dropped. 607 // 608 // In the vast majority of cases, only one loop iteration is needed; 609 // however, multiple concurrent calls to goroutineLeakGC could lead to 610 // the execution of additional GC cycles. 611 // 612 // Examples: 613 // 614 // pending? | G1 | G2 615 // ---------|-------------------------|----------------------- 616 // - | goroutineLeakGC() | goroutineLeakGC() 617 // - | pending.Store(true) | . 618 // X | for pending.Load() | . 619 // X | GC() | . 620 // X | > gcStart() | . 621 // X | pending.Store(false) | . 622 // ... 623 // - | > gcMarkDone() | . 624 // - | . | pending.Store(true) 625 // ... 626 // X | > gcMarkTermination() | . 627 // X | ... 628 // X | < GC returns | . 629 // X | for pending.Load | . 630 // X | GC() | . 631 // X | . | for pending.Load() 632 // X | . | GC() 633 // ... 634 // The first to pick up the pending flag will start a 635 // leak detection cycle. 636 for work.goroutineLeak.pending.Load() { 637 GC() 638 } 639 } 640 641 // gcWaitOnMark blocks until GC finishes the Nth mark phase. If GC has 642 // already completed this mark phase, it returns immediately. 643 func gcWaitOnMark(n uint32) { 644 for { 645 // Disable phase transitions. 646 lock(&work.sweepWaiters.lock) 647 nMarks := work.cycles.Load() 648 if gcphase != _GCmark { 649 // We've already completed this cycle's mark. 650 nMarks++ 651 } 652 if nMarks > n { 653 // We're done. 654 unlock(&work.sweepWaiters.lock) 655 return 656 } 657 658 // Wait until sweep termination, mark, and mark 659 // termination of cycle N complete. 660 work.sweepWaiters.list.push(getg()) 661 goparkunlock(&work.sweepWaiters.lock, waitReasonWaitForGCCycle, traceBlockUntilGCEnds, 1) 662 } 663 } 664 665 // gcMode indicates how concurrent a GC cycle should be. 666 type gcMode int 667 668 const ( 669 gcBackgroundMode gcMode = iota // concurrent GC and sweep 670 gcForceMode // stop-the-world GC now, concurrent sweep 671 gcForceBlockMode // stop-the-world GC now and STW sweep (forced by user) 672 ) 673 674 // A gcTrigger is a predicate for starting a GC cycle. Specifically, 675 // it is an exit condition for the _GCoff phase. 676 type gcTrigger struct { 677 kind gcTriggerKind 678 now int64 // gcTriggerTime: current time 679 n uint32 // gcTriggerCycle: cycle number to start 680 } 681 682 type gcTriggerKind int 683 684 const ( 685 // gcTriggerHeap indicates that a cycle should be started when 686 // the heap size reaches the trigger heap size computed by the 687 // controller. 688 gcTriggerHeap gcTriggerKind = iota 689 690 // gcTriggerTime indicates that a cycle should be started when 691 // it's been more than forcegcperiod nanoseconds since the 692 // previous GC cycle. 693 gcTriggerTime 694 695 // gcTriggerCycle indicates that a cycle should be started if 696 // we have not yet started cycle number gcTrigger.n (relative 697 // to work.cycles). 698 gcTriggerCycle 699 ) 700 701 // test reports whether the trigger condition is satisfied, meaning 702 // that the exit condition for the _GCoff phase has been met. The exit 703 // condition should be tested when allocating. 704 func (t gcTrigger) test() bool { 705 if !memstats.enablegc || panicking.Load() != 0 || gcphase != _GCoff { 706 return false 707 } 708 switch t.kind { 709 case gcTriggerHeap: 710 trigger, _ := gcController.trigger() 711 return gcController.heapLive.Load() >= trigger 712 case gcTriggerTime: 713 if gcController.gcPercent.Load() < 0 { 714 return false 715 } 716 lastgc := int64(atomic.Load64(&memstats.last_gc_nanotime)) 717 return lastgc != 0 && t.now-lastgc > forcegcperiod 718 case gcTriggerCycle: 719 // t.n > work.cycles, but accounting for wraparound. 720 return int32(t.n-work.cycles.Load()) > 0 721 } 722 return true 723 } 724 725 // gcStart starts the GC. It transitions from _GCoff to _GCmark (if 726 // debug.gcstoptheworld == 0) or performs all of GC (if 727 // debug.gcstoptheworld != 0). 728 // 729 // This may return without performing this transition in some cases, 730 // such as when called on a system stack or with locks held. 731 func gcStart(trigger gcTrigger) { 732 // Since this is called from malloc and malloc is called in 733 // the guts of a number of libraries that might be holding 734 // locks, don't attempt to start GC in non-preemptible or 735 // potentially unstable situations. 736 mp := acquirem() 737 if gp := getg(); gp == mp.g0 || mp.locks > 1 || mp.preemptoff != "" { 738 releasem(mp) 739 return 740 } 741 releasem(mp) 742 mp = nil 743 744 if gp := getg(); gp.bubble != nil { 745 // Disassociate the G from its synctest bubble while allocating. 746 // This is less elegant than incrementing the group's active count, 747 // but avoids any contamination between GC and synctest. 748 bubble := gp.bubble 749 gp.bubble = nil 750 defer func() { 751 gp.bubble = bubble 752 }() 753 } 754 755 // Pick up the remaining unswept/not being swept spans concurrently 756 // 757 // This shouldn't happen if we're being invoked in background 758 // mode since proportional sweep should have just finished 759 // sweeping everything, but rounding errors, etc, may leave a 760 // few spans unswept. In forced mode, this is necessary since 761 // GC can be forced at any point in the sweeping cycle. 762 // 763 // We check the transition condition continuously here in case 764 // this G gets delayed in to the next GC cycle. 765 for trigger.test() && sweepone() != ^uintptr(0) { 766 } 767 768 // Perform GC initialization and the sweep termination 769 // transition. 770 semacquire(&work.startSema) 771 // Re-check transition condition under transition lock. 772 if !trigger.test() { 773 semrelease(&work.startSema) 774 return 775 } 776 777 // In gcstoptheworld debug mode, upgrade the mode accordingly. 778 // We do this after re-checking the transition condition so 779 // that multiple goroutines that detect the heap trigger don't 780 // start multiple STW GCs. 781 mode := gcBackgroundMode 782 if debug.gcstoptheworld == 1 { 783 mode = gcForceMode 784 } else if debug.gcstoptheworld == 2 { 785 mode = gcForceBlockMode 786 } 787 788 // Ok, we're doing it! Stop everybody else 789 semacquire(&gcsema) 790 semacquire(&worldsema) 791 792 // For stats, check if this GC was forced by the user. 793 // Update it under gcsema to avoid gctrace getting wrong values. 794 work.userForced = trigger.kind == gcTriggerCycle 795 796 trace := traceAcquire() 797 if trace.ok() { 798 trace.GCStart() 799 traceRelease(trace) 800 } 801 802 // Check and setup per-P state. 803 for _, p := range allp { 804 // Check that all Ps have finished deferred mcache flushes. 805 if fg := p.mcache.flushGen.Load(); fg != mheap_.sweepgen { 806 println("runtime: p", p.id, "flushGen", fg, "!= sweepgen", mheap_.sweepgen) 807 throw("p mcache not flushed") 808 } 809 // Initialize ptrBuf if necessary. 810 if goexperiment.GreenTeaGC && p.gcw.ptrBuf == nil { 811 p.gcw.ptrBuf = (*[gc.PageSize / goarch.PtrSize]uintptr)(persistentalloc(gc.PageSize, goarch.PtrSize, &memstats.gcMiscSys)) 812 } 813 } 814 815 gcBgMarkStartWorkers() 816 817 systemstack(gcResetMarkState) 818 819 work.stwprocs, work.maxprocs = gomaxprocs, gomaxprocs 820 if work.stwprocs > numCPUStartup { 821 // This is used to compute CPU time of the STW phases, so it 822 // can't be more than the CPU count, even if GOMAXPROCS is. 823 work.stwprocs = numCPUStartup 824 } 825 work.heap0 = gcController.heapLive.Load() 826 work.pauseNS = 0 827 work.mode = mode 828 829 now := nanotime() 830 work.tSweepTerm = now 831 var stw worldStop 832 systemstack(func() { 833 stw = stopTheWorldWithSema(stwGCSweepTerm) 834 }) 835 836 // Accumulate fine-grained stopping time. 837 work.cpuStats.accumulateGCPauseTime(stw.stoppingCPUTime, 1) 838 839 // Finish sweep before we start concurrent scan. 840 systemstack(func() { 841 finishsweep_m() 842 }) 843 844 // clearpools before we start the GC. If we wait the memory will not be 845 // reclaimed until the next GC cycle. 846 clearpools() 847 848 work.cycles.Add(1) 849 850 // Assists and workers can start the moment we start 851 // the world. 852 gcController.startCycle(now, int(gomaxprocs), trigger) 853 854 // Notify the CPU limiter that assists may begin. 855 gcCPULimiter.startGCTransition(true, now) 856 857 // In STW mode, disable scheduling of user Gs. This may also 858 // disable scheduling of this goroutine, so it may block as 859 // soon as we start the world again. 860 if mode != gcBackgroundMode { 861 schedEnableUser(false) 862 } 863 864 // If goroutine leak detection is pending, enable it for this GC cycle. 865 if work.goroutineLeak.pending.Load() { 866 work.goroutineLeak.enabled = true 867 work.goroutineLeak.pending.Store(false) 868 // Set all sync objects of blocked goroutines as untraceable 869 // by the GC. Only set as traceable at the end of the GC cycle. 870 setSyncObjectsUntraceable() 871 } 872 873 // Enter concurrent mark phase and enable 874 // write barriers. 875 // 876 // Because the world is stopped, all Ps will 877 // observe that write barriers are enabled by 878 // the time we start the world and begin 879 // scanning. 880 // 881 // Write barriers must be enabled before assists are 882 // enabled because they must be enabled before 883 // any non-leaf heap objects are marked. Since 884 // allocations are blocked until assists can 885 // happen, we want to enable assists as early as 886 // possible. 887 setGCPhase(_GCmark) 888 889 gcBgMarkPrepare() // Must happen before assists are enabled. 890 gcPrepareMarkRoots() 891 892 // Mark all active tinyalloc blocks. Since we're 893 // allocating from these, they need to be black like 894 // other allocations. The alternative is to blacken 895 // the tiny block on every allocation from it, which 896 // would slow down the tiny allocator. 897 gcMarkTinyAllocs() 898 899 // At this point all Ps have enabled the write 900 // barrier, thus maintaining the no white to 901 // black invariant. Enable mutator assists to 902 // put back-pressure on fast allocating 903 // mutators. 904 atomic.Store(&gcBlackenEnabled, 1) 905 906 // In STW mode, we could block the instant systemstack 907 // returns, so make sure we're not preemptible. 908 mp = acquirem() 909 910 // Update the CPU stats pause time. 911 // 912 // Use maxprocs instead of stwprocs here because the total time 913 // computed in the CPU stats is based on maxprocs, and we want them 914 // to be comparable. 915 work.cpuStats.accumulateGCPauseTime(nanotime()-stw.finishedStopping, work.maxprocs) 916 917 // Concurrent mark. 918 systemstack(func() { 919 now = startTheWorldWithSema(0, stw) 920 work.pauseNS += now - stw.startedStopping 921 work.tMark = now 922 923 // Release the CPU limiter. 924 gcCPULimiter.finishGCTransition(now) 925 }) 926 927 // Release the world sema before Gosched() in STW mode 928 // because we will need to reacquire it later but before 929 // this goroutine becomes runnable again, and we could 930 // self-deadlock otherwise. 931 semrelease(&worldsema) 932 releasem(mp) 933 934 // Make sure we block instead of returning to user code 935 // in STW mode. 936 if mode != gcBackgroundMode { 937 Gosched() 938 } 939 940 semrelease(&work.startSema) 941 } 942 943 // gcMarkDoneFlushed counts the number of P's with flushed work. 944 // 945 // Ideally this would be a captured local in gcMarkDone, but forEachP 946 // escapes its callback closure, so it can't capture anything. 947 // 948 // This is protected by markDoneSema. 949 var gcMarkDoneFlushed uint32 950 951 // gcDebugMarkDone contains fields used to debug/test mark termination. 952 var gcDebugMarkDone struct { 953 // spinAfterRaggedBarrier forces gcMarkDone to spin after it executes 954 // the ragged barrier. 955 spinAfterRaggedBarrier atomic.Bool 956 957 // restartedDueTo27993 indicates that we restarted mark termination 958 // due to the bug described in issue #27993. 959 // 960 // Protected by worldsema. 961 restartedDueTo27993 bool 962 } 963 964 // gcMarkDone transitions the GC from mark to mark termination if all 965 // reachable objects have been marked (that is, there are no grey 966 // objects and can be no more in the future). Otherwise, it flushes 967 // all local work to the global queues where it can be discovered by 968 // other workers. 969 // 970 // All goroutines performing GC work must call gcBeginWork to signal 971 // that they're executing GC work. They must call gcEndWork when done. 972 // This should be called when all local mark work has been drained and 973 // there are no remaining workers. Specifically, when gcEndWork returns 974 // true. 975 // 976 // The calling context must be preemptible. 977 // 978 // Flushing local work is important because idle Ps may have local 979 // work queued. This is the only way to make that work visible and 980 // drive GC to completion. 981 // 982 // It is explicitly okay to have write barriers in this function. If 983 // it does transition to mark termination, then all reachable objects 984 // have been marked, so the write barrier cannot shade any more 985 // objects. 986 func gcMarkDone() { 987 // Ensure only one thread is running the ragged barrier at a 988 // time. 989 semacquire(&work.markDoneSema) 990 991 top: 992 // Re-check transition condition under transition lock. 993 // 994 // It's critical that this checks the global work queues are 995 // empty before performing the ragged barrier. Otherwise, 996 // there could be global work that a P could take after the P 997 // has passed the ragged barrier. 998 if !(gcphase == _GCmark && gcIsMarkDone()) { 999 semrelease(&work.markDoneSema) 1000 return 1001 } 1002 1003 // forEachP needs worldsema to execute, and we'll need it to 1004 // stop the world later, so acquire worldsema now. 1005 semacquire(&worldsema) 1006 1007 // Prevent weak->strong conversions from generating additional 1008 // GC work. forEachP will guarantee that it is observed globally. 1009 work.strongFromWeak.block = true 1010 1011 // Flush all local buffers and collect flushedWork flags. 1012 gcMarkDoneFlushed = 0 1013 forEachP(waitReasonGCMarkTermination, func(pp *p) { 1014 // Flush the write barrier buffer, since this may add 1015 // work to the gcWork. 1016 wbBufFlush1(pp) 1017 1018 // Flush the gcWork, since this may create global work 1019 // and set the flushedWork flag. 1020 // 1021 // TODO(austin): Break up these workbufs to 1022 // better distribute work. 1023 pp.gcw.dispose() 1024 1025 // Collect the flushedWork flag. 1026 if pp.gcw.flushedWork { 1027 atomic.Xadd(&gcMarkDoneFlushed, 1) 1028 pp.gcw.flushedWork = false 1029 } 1030 }) 1031 1032 if gcMarkDoneFlushed != 0 { 1033 // More grey objects were discovered since the 1034 // previous termination check, so there may be more 1035 // work to do. Keep going. It's possible the 1036 // transition condition became true again during the 1037 // ragged barrier, so re-check it. 1038 semrelease(&worldsema) 1039 goto top 1040 } 1041 1042 // For debugging/testing. 1043 for gcDebugMarkDone.spinAfterRaggedBarrier.Load() { 1044 } 1045 1046 // There was no global work, no local work, and no Ps 1047 // communicated work since we took markDoneSema. Therefore 1048 // there are no grey objects and no more objects can be 1049 // shaded. Transition to mark termination. 1050 now := nanotime() 1051 work.tMarkTerm = now 1052 getg().m.preemptoff = "gcing" 1053 var stw worldStop 1054 systemstack(func() { 1055 stw = stopTheWorldWithSema(stwGCMarkTerm) 1056 }) 1057 // The gcphase is _GCmark, it will transition to _GCmarktermination 1058 // below. The important thing is that the wb remains active until 1059 // all marking is complete. This includes writes made by the GC. 1060 1061 // Accumulate fine-grained stopping time. 1062 work.cpuStats.accumulateGCPauseTime(stw.stoppingCPUTime, 1) 1063 1064 // There is sometimes work left over when we enter mark termination due 1065 // to write barriers performed after the completion barrier above. 1066 // Detect this and resume concurrent mark. This is obviously 1067 // unfortunate. 1068 // 1069 // See issue #27993 for details. 1070 // 1071 // Switch to the system stack to call wbBufFlush1, though in this case 1072 // it doesn't matter because we're non-preemptible anyway. 1073 restart := false 1074 systemstack(func() { 1075 for _, p := range allp { 1076 wbBufFlush1(p) 1077 if !p.gcw.empty() { 1078 restart = true 1079 break 1080 } 1081 } 1082 }) 1083 1084 // Check whether we need to resume the marking phase because of issue #27993 1085 // or because of goroutine leak detection. 1086 if restart || (work.goroutineLeak.enabled && !work.goroutineLeak.done) { 1087 if restart { 1088 // Restart because of issue #27993. 1089 gcDebugMarkDone.restartedDueTo27993 = true 1090 } else { 1091 // Marking has reached a fixed-point. Attempt to detect goroutine leaks. 1092 // 1093 // If the returned value is true, then detection already concluded for this cycle. 1094 // Otherwise, more runnable goroutines were discovered, requiring additional mark work. 1095 work.goroutineLeak.done = findGoroutineLeaks() 1096 } 1097 1098 getg().m.preemptoff = "" 1099 systemstack(func() { 1100 // Accumulate the time we were stopped before we had to start again. 1101 work.cpuStats.accumulateGCPauseTime(nanotime()-stw.finishedStopping, work.maxprocs) 1102 1103 // Start the world again. 1104 now := startTheWorldWithSema(0, stw) 1105 work.pauseNS += now - stw.startedStopping 1106 }) 1107 semrelease(&worldsema) 1108 goto top 1109 } 1110 1111 gcComputeStartingStackSize() 1112 1113 // Disable assists and background workers. We must do 1114 // this before waking blocked assists. 1115 atomic.Store(&gcBlackenEnabled, 0) 1116 1117 // Notify the CPU limiter that GC assists will now cease. 1118 gcCPULimiter.startGCTransition(false, now) 1119 1120 // Wake all blocked assists. These will run when we 1121 // start the world again. 1122 gcWakeAllAssists() 1123 1124 // Wake all blocked weak->strong conversions. These will run 1125 // when we start the world again. 1126 work.strongFromWeak.block = false 1127 gcWakeAllStrongFromWeak() 1128 1129 // Likewise, release the transition lock. Blocked 1130 // workers and assists will run when we start the 1131 // world again. 1132 semrelease(&work.markDoneSema) 1133 1134 // In STW mode, re-enable user goroutines. These will be 1135 // queued to run after we start the world. 1136 schedEnableUser(true) 1137 1138 // endCycle depends on all gcWork cache stats being flushed. 1139 // The termination algorithm above ensured that up to 1140 // allocations since the ragged barrier. 1141 gcController.endCycle(now, int(gomaxprocs), work.userForced) 1142 1143 // Perform mark termination. This will restart the world. 1144 gcMarkTermination(stw) 1145 } 1146 1147 // isMaybeRunnable checks whether a goroutine may still be semantically runnable. 1148 // For goroutines which are semantically runnable, this will eventually return true 1149 // as the GC marking phase progresses. It returns false for leaked goroutines, or for 1150 // goroutines which are not yet computed as possibly runnable by the GC. 1151 func (gp *g) isMaybeRunnable() bool { 1152 // Check whether the goroutine is actually in a waiting state first. 1153 if readgstatus(gp) != _Gwaiting { 1154 // If the goroutine is not waiting, then clearly it is maybe runnable. 1155 return true 1156 } 1157 1158 switch gp.waitreason { 1159 case waitReasonSelectNoCases, 1160 waitReasonChanSendNilChan, 1161 waitReasonChanReceiveNilChan: 1162 // Select with no cases or communicating on nil channels 1163 // make goroutines unrunnable by definition. 1164 return false 1165 case waitReasonChanReceive, 1166 waitReasonSelect, 1167 waitReasonChanSend: 1168 // Cycle all through all *sudog to check whether 1169 // the goroutine is waiting on a marked channel. 1170 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1171 if isMarkedOrNotInHeap(unsafe.Pointer(sg.c.get())) { 1172 return true 1173 } 1174 } 1175 return false 1176 case waitReasonSyncCondWait, 1177 waitReasonSyncWaitGroupWait, 1178 waitReasonSyncMutexLock, 1179 waitReasonSyncRWMutexLock, 1180 waitReasonSyncRWMutexRLock: 1181 // If waiting on mutexes, wait groups, or condition variables, 1182 // check if the synchronization primitive attached to the sudog is marked. 1183 if gp.waiting != nil { 1184 return isMarkedOrNotInHeap(gp.waiting.elem.get()) 1185 } 1186 } 1187 return true 1188 } 1189 1190 // findMaybeRunnableGoroutines checks to see if more blocked but maybe-runnable goroutines exist. 1191 // If so, it adds them into root set and increments work.markrootJobs accordingly. 1192 // Returns true if we need to run another phase of markroots; returns false otherwise. 1193 func findMaybeRunnableGoroutines() (moreWork bool) { 1194 oldRootJobs := work.markrootJobs.Load() 1195 1196 // To begin with we have a set of unchecked stackRoots between 1197 // vIndex and ivIndex. During the loop, anything < vIndex should be 1198 // valid stackRoots and anything >= ivIndex should be invalid stackRoots. 1199 // The loop terminates when the two indices meet. 1200 var vIndex, ivIndex int = work.nMaybeRunnableStackRoots, work.nStackRoots 1201 // Reorder goroutine list 1202 for vIndex < ivIndex { 1203 if work.stackRoots[vIndex].isMaybeRunnable() { 1204 vIndex = vIndex + 1 1205 continue 1206 } 1207 for ivIndex = ivIndex - 1; ivIndex != vIndex; ivIndex = ivIndex - 1 { 1208 if gp := work.stackRoots[ivIndex]; gp.isMaybeRunnable() { 1209 work.stackRoots[ivIndex] = work.stackRoots[vIndex] 1210 work.stackRoots[vIndex] = gp 1211 vIndex = vIndex + 1 1212 break 1213 } 1214 } 1215 } 1216 1217 newRootJobs := work.baseStacks + uint32(vIndex) 1218 if newRootJobs > oldRootJobs { 1219 work.nMaybeRunnableStackRoots = vIndex 1220 work.markrootJobs.Store(newRootJobs) 1221 } 1222 return newRootJobs > oldRootJobs 1223 } 1224 1225 // setSyncObjectsUntraceable scans allgs and sets the elem and c fields of all sudogs to 1226 // an untrackable pointer. This prevents the GC from marking these objects as live in memory 1227 // by following these pointers when runnning deadlock detection. 1228 func setSyncObjectsUntraceable() { 1229 assertWorldStopped() 1230 1231 forEachGRace(func(gp *g) { 1232 // Set as untraceable all synchronization objects of goroutines 1233 // blocked at concurrency operations that could leak. 1234 switch { 1235 case gp.waitreason.isSyncWait(): 1236 // Synchronization primitives are reachable from the *sudog via 1237 // via the elem field. 1238 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1239 sg.elem.setUntraceable() 1240 } 1241 case gp.waitreason.isChanWait(): 1242 // Channels and select statements are reachable from the *sudog via the c field. 1243 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1244 sg.c.setUntraceable() 1245 } 1246 } 1247 }) 1248 } 1249 1250 // gcRestoreSyncObjects restores the elem and c fields of all sudogs to their original values. 1251 // Should be invoked after the goroutine leak detection phase. 1252 func gcRestoreSyncObjects() { 1253 assertWorldStopped() 1254 1255 forEachGRace(func(gp *g) { 1256 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1257 sg.elem.setTraceable() 1258 sg.c.setTraceable() 1259 } 1260 }) 1261 } 1262 1263 // findGoroutineLeaks scans the remaining stackRoots and marks any which are 1264 // blocked over exclusively unreachable concurrency primitives as leaked (deadlocked). 1265 // Returns true if the goroutine leak check was performed (or unnecessary). 1266 // Returns false if the GC cycle has not yet computed all maybe-runnable goroutines. 1267 func findGoroutineLeaks() bool { 1268 assertWorldStopped() 1269 1270 // Report goroutine leaks and mark them unreachable, and resume marking 1271 // we still need to mark these unreachable *g structs as they 1272 // get reused, but their stack won't get scanned 1273 if work.nMaybeRunnableStackRoots == work.nStackRoots { 1274 // nMaybeRunnableStackRoots == nStackRoots means that all goroutines are marked. 1275 return true 1276 } 1277 1278 // Check whether any more maybe-runnable goroutines can be found by the GC. 1279 if findMaybeRunnableGoroutines() { 1280 // We found more work, so we need to resume the marking phase. 1281 return false 1282 } 1283 1284 // For the remaining goroutines, mark them as unreachable and leaked. 1285 work.goroutineLeak.count = work.nStackRoots - work.nMaybeRunnableStackRoots 1286 1287 for i := work.nMaybeRunnableStackRoots; i < work.nStackRoots; i++ { 1288 gp := work.stackRoots[i] 1289 casgstatus(gp, _Gwaiting, _Gleaked) 1290 1291 // Add the primitives causing the goroutine leaks 1292 // to the GC work queue, to ensure they are marked. 1293 // 1294 // NOTE(vsaioc): these primitives should also be reachable 1295 // from the goroutine's stack, but let's play it safe. 1296 switch { 1297 case gp.waitreason.isChanWait(): 1298 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1299 shade(sg.c.uintptr()) 1300 } 1301 case gp.waitreason.isSyncWait(): 1302 for sg := gp.waiting; sg != nil; sg = sg.waitlink { 1303 shade(sg.elem.uintptr()) 1304 } 1305 } 1306 } 1307 // Put the remaining roots as ready for marking and drain them. 1308 work.markrootJobs.Add(int32(work.nStackRoots - work.nMaybeRunnableStackRoots)) 1309 work.nMaybeRunnableStackRoots = work.nStackRoots 1310 return true 1311 } 1312 1313 // World must be stopped and mark assists and background workers must be 1314 // disabled. 1315 func gcMarkTermination(stw worldStop) { 1316 // Start marktermination (write barrier remains enabled for now). 1317 setGCPhase(_GCmarktermination) 1318 1319 work.heap1 = gcController.heapLive.Load() 1320 startTime := nanotime() 1321 1322 mp := acquirem() 1323 mp.preemptoff = "gcing" 1324 mp.traceback = 2 1325 curgp := mp.curg 1326 // N.B. The execution tracer is not aware of this status 1327 // transition and handles it specially based on the 1328 // wait reason. 1329 casGToWaitingForSuspendG(curgp, _Grunning, waitReasonGarbageCollection) 1330 1331 // Run gc on the g0 stack. We do this so that the g stack 1332 // we're currently running on will no longer change. Cuts 1333 // the root set down a bit (g0 stacks are not scanned, and 1334 // we don't need to scan gc's internal state). We also 1335 // need to switch to g0 so we can shrink the stack. 1336 systemstack(func() { 1337 gcMark(startTime) 1338 // Must return immediately. 1339 // The outer function's stack may have moved 1340 // during gcMark (it shrinks stacks, including the 1341 // outer function's stack), so we must not refer 1342 // to any of its variables. Return back to the 1343 // non-system stack to pick up the new addresses 1344 // before continuing. 1345 }) 1346 1347 var stwSwept bool 1348 systemstack(func() { 1349 work.heap2 = work.bytesMarked 1350 if debug.gccheckmark > 0 { 1351 runCheckmark(func(_ *gcWork) { gcPrepareMarkRoots() }) 1352 } 1353 if debug.checkfinalizers > 0 { 1354 checkFinalizersAndCleanups() 1355 } 1356 1357 // marking is complete so we can turn the write barrier off 1358 setGCPhase(_GCoff) 1359 stwSwept = gcSweep(work.mode) 1360 }) 1361 1362 mp.traceback = 0 1363 casgstatus(curgp, _Gwaiting, _Grunning) 1364 1365 trace := traceAcquire() 1366 if trace.ok() { 1367 trace.GCDone() 1368 traceRelease(trace) 1369 } 1370 1371 // all done 1372 mp.preemptoff = "" 1373 1374 if gcphase != _GCoff { 1375 throw("gc done but gcphase != _GCoff") 1376 } 1377 1378 // Record heapInUse for scavenger. 1379 memstats.lastHeapInUse = gcController.heapInUse.load() 1380 1381 // Update GC trigger and pacing, as well as downstream consumers 1382 // of this pacing information, for the next cycle. 1383 systemstack(gcControllerCommit) 1384 1385 // Update timing memstats 1386 now := nanotime() 1387 sec, nsec, _ := time_now() 1388 unixNow := sec*1e9 + int64(nsec) 1389 work.pauseNS += now - stw.startedStopping 1390 work.tEnd = now 1391 atomic.Store64(&memstats.last_gc_unix, uint64(unixNow)) // must be Unix time to make sense to user 1392 atomic.Store64(&memstats.last_gc_nanotime, uint64(now)) // monotonic time for us 1393 memstats.pause_ns[memstats.numgc%uint32(len(memstats.pause_ns))] = uint64(work.pauseNS) 1394 memstats.pause_end[memstats.numgc%uint32(len(memstats.pause_end))] = uint64(unixNow) 1395 memstats.pause_total_ns += uint64(work.pauseNS) 1396 1397 // Accumulate CPU stats. 1398 // 1399 // Use maxprocs instead of stwprocs for GC pause time because the total time 1400 // computed in the CPU stats is based on maxprocs, and we want them to be 1401 // comparable. 1402 // 1403 // Pass gcMarkPhase=true to accumulate so we can get all the latest GC CPU stats 1404 // in there too. 1405 work.cpuStats.accumulateGCPauseTime(now-stw.finishedStopping, work.maxprocs) 1406 work.cpuStats.accumulate(now, true) 1407 1408 // Compute overall GC CPU utilization. 1409 // Omit idle marking time from the overall utilization here since it's "free". 1410 memstats.gc_cpu_fraction = float64(work.cpuStats.GCTotalTime-work.cpuStats.GCIdleTime) / float64(work.cpuStats.TotalTime) 1411 1412 // Reset assist time and background time stats. 1413 // 1414 // Do this now, instead of at the start of the next GC cycle, because 1415 // these two may keep accumulating even if the GC is not active. 1416 scavenge.assistTime.Store(0) 1417 scavenge.backgroundTime.Store(0) 1418 1419 // Reset idle time stat. 1420 sched.idleTime.Store(0) 1421 1422 if work.userForced { 1423 memstats.numforcedgc++ 1424 } 1425 1426 // Bump GC cycle count and wake goroutines waiting on sweep. 1427 lock(&work.sweepWaiters.lock) 1428 memstats.numgc++ 1429 injectglist(&work.sweepWaiters.list) 1430 unlock(&work.sweepWaiters.lock) 1431 1432 // Increment the scavenge generation now. 1433 // 1434 // This moment represents peak heap in use because we're 1435 // about to start sweeping. 1436 mheap_.pages.scav.index.nextGen() 1437 1438 // Release the CPU limiter. 1439 gcCPULimiter.finishGCTransition(now) 1440 1441 // Finish the current heap profiling cycle and start a new 1442 // heap profiling cycle. We do this before starting the world 1443 // so events don't leak into the wrong cycle. 1444 mProf_NextCycle() 1445 1446 // There may be stale spans in mcaches that need to be swept. 1447 // Those aren't tracked in any sweep lists, so we need to 1448 // count them against sweep completion until we ensure all 1449 // those spans have been forced out. 1450 // 1451 // If gcSweep fully swept the heap (for example if the sweep 1452 // is not concurrent due to a GODEBUG setting), then we expect 1453 // the sweepLocker to be invalid, since sweeping is done. 1454 // 1455 // N.B. Below we might duplicate some work from gcSweep; this is 1456 // fine as all that work is idempotent within a GC cycle, and 1457 // we're still holding worldsema so a new cycle can't start. 1458 sl := sweep.active.begin() 1459 if !stwSwept && !sl.valid { 1460 throw("failed to set sweep barrier") 1461 } else if stwSwept && sl.valid { 1462 throw("non-concurrent sweep failed to drain all sweep queues") 1463 } 1464 1465 if work.goroutineLeak.enabled { 1466 // Restore the elem and c fields of all sudogs to their original values. 1467 gcRestoreSyncObjects() 1468 } 1469 1470 var goroutineLeakDone bool 1471 systemstack(func() { 1472 // Pull the GC out of goroutine leak detection mode. 1473 work.goroutineLeak.enabled = false 1474 goroutineLeakDone = work.goroutineLeak.done 1475 work.goroutineLeak.done = false 1476 1477 // The memstats updated above must be updated with the world 1478 // stopped to ensure consistency of some values, such as 1479 // sched.idleTime and sched.totaltime. memstats also include 1480 // the pause time (work,pauseNS), forcing computation of the 1481 // total pause time before the pause actually ends. 1482 // 1483 // Here we reuse the same now for start the world so that the 1484 // time added to /sched/pauses/total/gc:seconds will be 1485 // consistent with the value in memstats. 1486 startTheWorldWithSema(now, stw) 1487 }) 1488 1489 // Flush the heap profile so we can start a new cycle next GC. 1490 // This is relatively expensive, so we don't do it with the 1491 // world stopped. 1492 mProf_Flush() 1493 1494 // Prepare workbufs for freeing by the sweeper. We do this 1495 // asynchronously because it can take non-trivial time. 1496 prepareFreeWorkbufs() 1497 1498 // Free stack spans. This must be done between GC cycles. 1499 systemstack(freeStackSpans) 1500 1501 // Ensure all mcaches are flushed. Each P will flush its own 1502 // mcache before allocating, but idle Ps may not. Since this 1503 // is necessary to sweep all spans, we need to ensure all 1504 // mcaches are flushed before we start the next GC cycle. 1505 // 1506 // While we're here, flush the page cache for idle Ps to avoid 1507 // having pages get stuck on them. These pages are hidden from 1508 // the scavenger, so in small idle heaps a significant amount 1509 // of additional memory might be held onto. 1510 // 1511 // Also, flush the pinner cache, to avoid leaking that memory 1512 // indefinitely. 1513 if debug.gctrace > 1 { 1514 clear(memstats.lastScanStats[:]) 1515 } 1516 forEachP(waitReasonFlushProcCaches, func(pp *p) { 1517 pp.mcache.prepareForSweep() 1518 if pp.status == _Pidle { 1519 systemstack(func() { 1520 lock(&mheap_.lock) 1521 pp.pcache.flush(&mheap_.pages) 1522 unlock(&mheap_.lock) 1523 }) 1524 } 1525 if debug.gctrace > 1 { 1526 pp.gcw.flushScanStats(&memstats.lastScanStats) 1527 } 1528 pp.pinnerCache = nil 1529 }) 1530 if sl.valid { 1531 // Now that we've swept stale spans in mcaches, they don't 1532 // count against unswept spans. 1533 // 1534 // Note: this sweepLocker may not be valid if sweeping had 1535 // already completed during the STW. See the corresponding 1536 // begin() call that produced sl. 1537 sweep.active.end(sl) 1538 } 1539 1540 // Print gctrace before dropping worldsema. As soon as we drop 1541 // worldsema another cycle could start and smash the stats 1542 // we're trying to print. 1543 if debug.gctrace > 0 { 1544 util := int(memstats.gc_cpu_fraction * 100) 1545 1546 var sbuf [24]byte 1547 printlock() 1548 print("gc ", memstats.numgc, 1549 " @", string(itoaDiv(sbuf[:], uint64(work.tSweepTerm-runtimeInitTime)/1e6, 3)), "s ", 1550 util, "%") 1551 if goroutineLeakDone { 1552 print(" (checking for goroutine leaks)") 1553 } 1554 print(": ") 1555 prev := work.tSweepTerm 1556 for i, ns := range []int64{work.tMark, work.tMarkTerm, work.tEnd} { 1557 if i != 0 { 1558 print("+") 1559 } 1560 print(string(fmtNSAsMS(sbuf[:], uint64(ns-prev)))) 1561 prev = ns 1562 } 1563 print(" ms clock, ") 1564 for i, ns := range []int64{ 1565 int64(work.stwprocs) * (work.tMark - work.tSweepTerm), 1566 gcController.assistTime.Load(), 1567 gcController.dedicatedMarkTime.Load() + gcController.fractionalMarkTime.Load(), 1568 gcController.idleMarkTime.Load(), 1569 int64(work.stwprocs) * (work.tEnd - work.tMarkTerm), 1570 } { 1571 if i == 2 || i == 3 { 1572 // Separate mark time components with /. 1573 print("/") 1574 } else if i != 0 { 1575 print("+") 1576 } 1577 print(string(fmtNSAsMS(sbuf[:], uint64(ns)))) 1578 } 1579 print(" ms cpu, ", 1580 work.heap0>>20, "->", work.heap1>>20, "->", work.heap2>>20, " MB, ", 1581 gcController.lastHeapGoal>>20, " MB goal, ", 1582 gcController.lastStackScan.Load()>>20, " MB stacks, ", 1583 gcController.globalsScan.Load()>>20, " MB globals, ", 1584 work.maxprocs, " P") 1585 if work.userForced { 1586 print(" (forced)") 1587 } 1588 print("\n") 1589 1590 if debug.gctrace > 1 { 1591 dumpScanStats() 1592 } 1593 printunlock() 1594 } 1595 1596 // Print finalizer/cleanup queue length. Like gctrace, do this before the next GC starts. 1597 // The fact that the next GC might start is not that problematic here, but acts as a convenient 1598 // lock on printing this information (so it cannot overlap with itself from the next GC cycle). 1599 if debug.checkfinalizers > 0 { 1600 fq, fe := finReadQueueStats() 1601 fn := max(int64(fq)-int64(fe), 0) 1602 1603 cq, ce := gcCleanups.readQueueStats() 1604 cn := max(int64(cq)-int64(ce), 0) 1605 1606 println("checkfinalizers: queue:", fn, "finalizers +", cn, "cleanups") 1607 } 1608 1609 // Set any arena chunks that were deferred to fault. 1610 lock(&userArenaState.lock) 1611 faultList := userArenaState.fault 1612 userArenaState.fault = nil 1613 unlock(&userArenaState.lock) 1614 for _, lc := range faultList { 1615 lc.mspan.setUserArenaChunkToFault() 1616 } 1617 1618 // Enable huge pages on some metadata if we cross a heap threshold. 1619 if gcController.heapGoal() > minHeapForMetadataHugePages { 1620 systemstack(func() { 1621 mheap_.enableMetadataHugePages() 1622 }) 1623 } 1624 1625 semrelease(&worldsema) 1626 semrelease(&gcsema) 1627 // Careful: another GC cycle may start now. 1628 1629 releasem(mp) 1630 mp = nil 1631 1632 // now that gc is done, kick off finalizer thread if needed 1633 if !concurrentSweep { 1634 // give the queued finalizers, if any, a chance to run 1635 Gosched() 1636 } 1637 } 1638 1639 // gcBgMarkStartWorkers prepares background mark worker goroutines. These 1640 // goroutines will not run until the mark phase, but they must be started while 1641 // the work is not stopped and from a regular G stack. The caller must hold 1642 // worldsema. 1643 func gcBgMarkStartWorkers() { 1644 // Background marking is performed by per-P G's. Ensure that each P has 1645 // a background GC G. 1646 // 1647 // Worker Gs don't exit if gomaxprocs is reduced. If it is raised 1648 // again, we can reuse the old workers; no need to create new workers. 1649 if gcBgMarkWorkerCount >= gomaxprocs { 1650 return 1651 } 1652 1653 // Increment mp.locks when allocating. We are called within gcStart, 1654 // and thus must not trigger another gcStart via an allocation. gcStart 1655 // bails when allocating with locks held, so simulate that for these 1656 // allocations. 1657 // 1658 // TODO(prattmic): cleanup gcStart to use a more explicit "in gcStart" 1659 // check for bailing. 1660 mp := acquirem() 1661 ready := make(chan struct{}, 1) 1662 releasem(mp) 1663 1664 for gcBgMarkWorkerCount < gomaxprocs { 1665 mp := acquirem() // See above, we allocate a closure here. 1666 go gcBgMarkWorker(ready) 1667 releasem(mp) 1668 1669 // N.B. we intentionally wait on each goroutine individually 1670 // rather than starting all in a batch and then waiting once 1671 // afterwards. By running one goroutine at a time, we can take 1672 // advantage of runnext to bounce back and forth between 1673 // workers and this goroutine. In an overloaded application, 1674 // this can reduce GC start latency by prioritizing these 1675 // goroutines rather than waiting on the end of the run queue. 1676 <-ready 1677 // The worker is now guaranteed to be added to the pool before 1678 // its P's next findRunnableGCWorker. 1679 1680 gcBgMarkWorkerCount++ 1681 } 1682 } 1683 1684 // gcBgMarkPrepare sets up state for background marking. 1685 // Mutator assists must not yet be enabled. 1686 func gcBgMarkPrepare() { 1687 // Background marking will stop when the work queues are empty 1688 // and there are no more workers (note that, since this is 1689 // concurrent, this may be a transient state, but mark 1690 // termination will clean it up). Between background workers 1691 // and assists, we don't really know how many workers there 1692 // will be, so we pretend to have an arbitrarily large number 1693 // of workers, almost all of which are "waiting". While a 1694 // worker is working it decrements nwait. If nproc == nwait, 1695 // there are no workers. 1696 work.nproc = ^uint32(0) 1697 work.nwait = ^uint32(0) 1698 } 1699 1700 // gcBgMarkWorkerNode is an entry in the gcBgMarkWorkerPool. It points to a single 1701 // gcBgMarkWorker goroutine. 1702 type gcBgMarkWorkerNode struct { 1703 // Unused workers are managed in a lock-free stack. This field must be first. 1704 node lfnode 1705 1706 // The g of this worker. 1707 gp guintptr 1708 1709 // Release this m on park. This is used to communicate with the unlock 1710 // function, which cannot access the G's stack. It is unused outside of 1711 // gcBgMarkWorker(). 1712 m muintptr 1713 } 1714 type gcBgMarkWorkerNodePadded struct { 1715 gcBgMarkWorkerNode 1716 pad [tagAlign - unsafe.Sizeof(gcBgMarkWorkerNode{}) - gcBgMarkWorkerNodeRedZoneSize]byte 1717 } 1718 1719 const gcBgMarkWorkerNodeRedZoneSize = (16 << 2) * asanenabledBit // redZoneSize(512) 1720 1721 func gcBgMarkWorker(ready chan struct{}) { 1722 gp := getg() 1723 1724 // We pass node to a gopark unlock function, so it can't be on 1725 // the stack (see gopark). Prevent deadlock from recursively 1726 // starting GC by disabling preemption. 1727 gp.m.preemptoff = "GC worker init" 1728 node := &new(gcBgMarkWorkerNodePadded).gcBgMarkWorkerNode // TODO: technically not allowed in the heap. See comment in tagptr.go. 1729 gp.m.preemptoff = "" 1730 1731 node.gp.set(gp) 1732 1733 node.m.set(acquirem()) 1734 1735 ready <- struct{}{} 1736 // After this point, the background mark worker is generally scheduled 1737 // cooperatively by gcController.findRunnableGCWorker. While performing 1738 // work on the P, preemption is disabled because we are working on 1739 // P-local work buffers. When the preempt flag is set, this puts itself 1740 // into _Gwaiting to be woken up by gcController.findRunnableGCWorker 1741 // at the appropriate time. 1742 // 1743 // When preemption is enabled (e.g., while in gcMarkDone), this worker 1744 // may be preempted and schedule as a _Grunnable G from a runq. That is 1745 // fine; it will eventually gopark again for further scheduling via 1746 // findRunnableGCWorker. 1747 // 1748 // Since we disable preemption before notifying ready, we guarantee that 1749 // this G will be in the worker pool for the next findRunnableGCWorker. 1750 // This isn't strictly necessary, but it reduces latency between 1751 // _GCmark starting and the workers starting. 1752 1753 for { 1754 // Go to sleep until woken by 1755 // gcController.findRunnableGCWorker. 1756 gopark(func(g *g, nodep unsafe.Pointer) bool { 1757 node := (*gcBgMarkWorkerNode)(nodep) 1758 1759 if mp := node.m.ptr(); mp != nil { 1760 // The worker G is no longer running; release 1761 // the M. 1762 // 1763 // N.B. it is _safe_ to release the M as soon 1764 // as we are no longer performing P-local mark 1765 // work. 1766 // 1767 // However, since we cooperatively stop work 1768 // when gp.preempt is set, if we releasem in 1769 // the loop then the following call to gopark 1770 // would immediately preempt the G. This is 1771 // also safe, but inefficient: the G must 1772 // schedule again only to enter gopark and park 1773 // again. Thus, we defer the release until 1774 // after parking the G. 1775 releasem(mp) 1776 } 1777 1778 // Release this G to the pool. 1779 gcBgMarkWorkerPool.push(&node.node) 1780 // Note that at this point, the G may immediately be 1781 // rescheduled and may be running. 1782 return true 1783 }, unsafe.Pointer(node), waitReasonGCWorkerIdle, traceBlockSystemGoroutine, 0) 1784 1785 // Preemption must not occur here, or another G might see 1786 // p.gcMarkWorkerMode. 1787 1788 // Disable preemption so we can use the gcw. If the 1789 // scheduler wants to preempt us, we'll stop draining, 1790 // dispose the gcw, and then preempt. 1791 node.m.set(acquirem()) 1792 pp := gp.m.p.ptr() // P can't change with preemption disabled. 1793 1794 if gcBlackenEnabled == 0 { 1795 println("worker mode", pp.gcMarkWorkerMode) 1796 throw("gcBgMarkWorker: blackening not enabled") 1797 } 1798 1799 if pp.gcMarkWorkerMode == gcMarkWorkerNotWorker { 1800 throw("gcBgMarkWorker: mode not set") 1801 } 1802 1803 startTime := nanotime() 1804 pp.gcMarkWorkerStartTime = startTime 1805 var trackLimiterEvent bool 1806 if pp.gcMarkWorkerMode == gcMarkWorkerIdleMode { 1807 trackLimiterEvent = pp.limiterEvent.start(limiterEventIdleMarkWork, startTime) 1808 } 1809 1810 gcBeginWork() 1811 1812 systemstack(func() { 1813 // Mark our goroutine preemptible so its stack can be scanned or observed 1814 // by the execution tracer. This, for example, lets two mark workers scan 1815 // each other (otherwise, they would deadlock). 1816 // 1817 // casGToWaitingForSuspendG marks the goroutine as ineligible for a 1818 // stack shrink, effectively pinning the stack in memory for the duration. 1819 // 1820 // N.B. The execution tracer is not aware of this status transition and 1821 // handles it specially based on the wait reason. 1822 casGToWaitingForSuspendG(gp, _Grunning, waitReasonGCWorkerActive) 1823 switch pp.gcMarkWorkerMode { 1824 default: 1825 throw("gcBgMarkWorker: unexpected gcMarkWorkerMode") 1826 case gcMarkWorkerDedicatedMode: 1827 gcDrainMarkWorkerDedicated(&pp.gcw, true) 1828 if gp.preempt { 1829 // We were preempted. This is 1830 // a useful signal to kick 1831 // everything out of the run 1832 // queue so it can run 1833 // somewhere else. 1834 if drainQ := runqdrain(pp); !drainQ.empty() { 1835 lock(&sched.lock) 1836 globrunqputbatch(&drainQ) 1837 unlock(&sched.lock) 1838 } 1839 } 1840 // Go back to draining, this time 1841 // without preemption. 1842 gcDrainMarkWorkerDedicated(&pp.gcw, false) 1843 case gcMarkWorkerFractionalMode: 1844 gcDrainMarkWorkerFractional(&pp.gcw) 1845 case gcMarkWorkerIdleMode: 1846 gcDrainMarkWorkerIdle(&pp.gcw) 1847 } 1848 casgstatus(gp, _Gwaiting, _Grunning) 1849 }) 1850 1851 // Account for time and mark us as stopped. 1852 now := nanotime() 1853 duration := now - startTime 1854 gcController.markWorkerStop(pp.gcMarkWorkerMode, duration) 1855 if trackLimiterEvent { 1856 pp.limiterEvent.stop(limiterEventIdleMarkWork, now) 1857 } 1858 if pp.gcMarkWorkerMode == gcMarkWorkerFractionalMode { 1859 atomic.Xaddint64(&pp.gcFractionalMarkTime, duration) 1860 } 1861 1862 // We'll releasem after this point and thus this P may run 1863 // something else. We must clear the worker mode to avoid 1864 // attributing the mode to a different (non-worker) G in 1865 // tracev2.GoStart. 1866 pp.gcMarkWorkerMode = gcMarkWorkerNotWorker 1867 1868 // If this worker reached a background mark completion 1869 // point, signal the main GC goroutine. 1870 if gcEndWork() { 1871 // We don't need the P-local buffers here, allow 1872 // preemption because we may schedule like a regular 1873 // goroutine in gcMarkDone (block on locks, etc). 1874 releasem(node.m.ptr()) 1875 node.m.set(nil) 1876 1877 gcMarkDone() 1878 } 1879 } 1880 } 1881 1882 // gcShouldScheduleWorker reports whether executing a mark worker 1883 // on p is potentially useful. p may be nil. 1884 func gcShouldScheduleWorker(p *p) bool { 1885 if p != nil && !p.gcw.empty() { 1886 return true 1887 } 1888 return gcMarkWorkAvailable() 1889 } 1890 1891 // gcIsMarkDone reports whether the mark phase is (probably) done. 1892 func gcIsMarkDone() bool { 1893 return work.nwait == work.nproc && !gcMarkWorkAvailable() 1894 } 1895 1896 // gcBeginWork signals to the garbage collector that a new worker is 1897 // about to process GC work. 1898 func gcBeginWork() { 1899 decnwait := atomic.Xadd(&work.nwait, -1) 1900 if decnwait == work.nproc { 1901 println("runtime: work.nwait=", decnwait, "work.nproc=", work.nproc) 1902 throw("work.nwait was > work.nproc") 1903 } 1904 } 1905 1906 // gcEndWork signals to the garbage collector that a new worker has just finished 1907 // its work. It reports whether it was the last worker and there's no more work 1908 // to do. If it returns true, the caller must call gcMarkDone. 1909 func gcEndWork() (last bool) { 1910 incnwait := atomic.Xadd(&work.nwait, +1) 1911 if incnwait > work.nproc { 1912 println("runtime: work.nwait=", incnwait, "work.nproc=", work.nproc) 1913 throw("work.nwait > work.nproc") 1914 } 1915 return incnwait == work.nproc && !gcMarkWorkAvailable() 1916 } 1917 1918 // gcMark runs the mark (or, for concurrent GC, mark termination) 1919 // All gcWork caches must be empty. 1920 // STW is in effect at this point. 1921 func gcMark(startTime int64) { 1922 if gcphase != _GCmarktermination { 1923 throw("in gcMark expecting to see gcphase as _GCmarktermination") 1924 } 1925 work.tstart = startTime 1926 1927 // Check that there's no marking work remaining. 1928 if next, jobs := work.markrootNext.Load(), work.markrootJobs.Load(); work.full != 0 || next < jobs { 1929 print("runtime: full=", hex(work.full), " next=", next, " jobs=", jobs, " nDataRoots=", work.nDataRoots, " nBSSRoots=", work.nBSSRoots, " nSpanRoots=", work.nSpanRoots, " nStackRoots=", work.nStackRoots, "\n") 1930 panic("non-empty mark queue after concurrent mark") 1931 } 1932 1933 if debug.gccheckmark > 0 { 1934 // This is expensive when there's a large number of 1935 // Gs, so only do it if checkmark is also enabled. 1936 gcMarkRootCheck() 1937 } 1938 1939 // Drop allg snapshot. allgs may have grown, in which case 1940 // this is the only reference to the old backing store and 1941 // there's no need to keep it around. 1942 work.stackRoots = nil 1943 1944 // Clear out buffers and double-check that all gcWork caches 1945 // are empty. This should be ensured by gcMarkDone before we 1946 // enter mark termination. 1947 // 1948 // TODO: We could clear out buffers just before mark if this 1949 // has a non-negligible impact on STW time. 1950 for _, p := range allp { 1951 // The write barrier may have buffered pointers since 1952 // the gcMarkDone barrier. However, since the barrier 1953 // ensured all reachable objects were marked, all of 1954 // these must be pointers to black objects. Hence we 1955 // can just discard the write barrier buffer. 1956 if debug.gccheckmark > 0 { 1957 // For debugging, flush the buffer and make 1958 // sure it really was all marked. 1959 wbBufFlush1(p) 1960 } else { 1961 p.wbBuf.reset() 1962 } 1963 1964 gcw := &p.gcw 1965 if !gcw.empty() { 1966 printlock() 1967 print("runtime: P ", p.id, " flushedWork ", gcw.flushedWork) 1968 if gcw.wbuf1 == nil { 1969 print(" wbuf1=<nil>") 1970 } else { 1971 print(" wbuf1.n=", gcw.wbuf1.nobj) 1972 } 1973 if gcw.wbuf2 == nil { 1974 print(" wbuf2=<nil>") 1975 } else { 1976 print(" wbuf2.n=", gcw.wbuf2.nobj) 1977 } 1978 print("\n") 1979 throw("P has cached GC work at end of mark termination") 1980 } 1981 // There may still be cached empty buffers, which we 1982 // need to flush since we're going to free them. Also, 1983 // there may be non-zero stats because we allocated 1984 // black after the gcMarkDone barrier. 1985 gcw.dispose() 1986 } 1987 1988 // Flush scanAlloc from each mcache since we're about to modify 1989 // heapScan directly. If we were to flush this later, then scanAlloc 1990 // might have incorrect information. 1991 // 1992 // Note that it's not important to retain this information; we know 1993 // exactly what heapScan is at this point via scanWork. 1994 for _, p := range allp { 1995 c := p.mcache 1996 if c == nil { 1997 continue 1998 } 1999 c.scanAlloc = 0 2000 } 2001 2002 // Reset controller state. 2003 gcController.resetLive(work.bytesMarked) 2004 } 2005 2006 // gcSweep must be called on the system stack because it acquires the heap 2007 // lock. See mheap for details. 2008 // 2009 // Returns true if the heap was fully swept by this function. 2010 // 2011 // The world must be stopped. 2012 // 2013 //go:systemstack 2014 func gcSweep(mode gcMode) bool { 2015 assertWorldStopped() 2016 2017 if gcphase != _GCoff { 2018 throw("gcSweep being done but phase is not GCoff") 2019 } 2020 2021 lock(&mheap_.lock) 2022 mheap_.sweepgen += 2 2023 sweep.active.reset() 2024 mheap_.pagesSwept.Store(0) 2025 mheap_.sweepArenas = mheap_.heapArenas 2026 mheap_.reclaimIndex.Store(0) 2027 mheap_.reclaimCredit.Store(0) 2028 unlock(&mheap_.lock) 2029 2030 sweep.centralIndex.clear() 2031 2032 if !concurrentSweep || mode == gcForceBlockMode { 2033 // Special case synchronous sweep. 2034 // Record that no proportional sweeping has to happen. 2035 lock(&mheap_.lock) 2036 mheap_.sweepPagesPerByte = 0 2037 unlock(&mheap_.lock) 2038 // Flush all mcaches. 2039 for _, pp := range allp { 2040 pp.mcache.prepareForSweep() 2041 } 2042 // Sweep all spans eagerly. 2043 for sweepone() != ^uintptr(0) { 2044 } 2045 // Free workbufs and span rings eagerly. 2046 prepareFreeWorkbufs() 2047 for freeSomeWbufs(false) { 2048 } 2049 freeDeadSpanSPMCs() 2050 // All "free" events for this mark/sweep cycle have 2051 // now happened, so we can make this profile cycle 2052 // available immediately. 2053 mProf_NextCycle() 2054 mProf_Flush() 2055 return true 2056 } 2057 2058 // Background sweep. 2059 lock(&sweep.lock) 2060 if sweep.parked { 2061 sweep.parked = false 2062 ready(sweep.g, 0, true) 2063 } 2064 unlock(&sweep.lock) 2065 return false 2066 } 2067 2068 // gcResetMarkState resets global state prior to marking (concurrent 2069 // or STW) and resets the stack scan state of all Gs. 2070 // 2071 // This is safe to do without the world stopped because any Gs created 2072 // during or after this will start out in the reset state. 2073 // 2074 // gcResetMarkState must be called on the system stack because it acquires 2075 // the heap lock. See mheap for details. 2076 // 2077 //go:systemstack 2078 func gcResetMarkState() { 2079 // This may be called during a concurrent phase, so lock to make sure 2080 // allgs doesn't change. 2081 forEachG(func(gp *g) { 2082 gp.gcscandone = false // set to true in gcphasework 2083 gp.gcAssistBytes = 0 2084 }) 2085 2086 // Clear page marks. This is just 1MB per 64GB of heap, so the 2087 // time here is pretty trivial. 2088 lock(&mheap_.lock) 2089 arenas := mheap_.heapArenas 2090 unlock(&mheap_.lock) 2091 for _, ai := range arenas { 2092 ha := mheap_.arenas[ai.l1()][ai.l2()] 2093 clear(ha.pageMarks[:]) 2094 } 2095 2096 work.bytesMarked = 0 2097 work.initialHeapLive = gcController.heapLive.Load() 2098 } 2099 2100 // Hooks for other packages 2101 2102 var poolcleanup func() 2103 var boringCaches []unsafe.Pointer // for crypto/internal/boring 2104 2105 // sync_runtime_registerPoolCleanup should be an internal detail, 2106 // but widely used packages access it using linkname. 2107 // Notable members of the hall of shame include: 2108 // - github.com/bytedance/gopkg 2109 // - github.com/songzhibin97/gkit 2110 // 2111 // Do not remove or change the type signature. 2112 // See go.dev/issue/67401. 2113 // 2114 //go:linkname sync_runtime_registerPoolCleanup sync.runtime_registerPoolCleanup 2115 func sync_runtime_registerPoolCleanup(f func()) { 2116 poolcleanup = f 2117 } 2118 2119 //go:linkname boring_registerCache crypto/internal/boring/bcache.registerCache 2120 func boring_registerCache(p unsafe.Pointer) { 2121 boringCaches = append(boringCaches, p) 2122 } 2123 2124 func clearpools() { 2125 // clear sync.Pools 2126 if poolcleanup != nil { 2127 poolcleanup() 2128 } 2129 2130 // clear boringcrypto caches 2131 for _, p := range boringCaches { 2132 atomicstorep(p, nil) 2133 } 2134 2135 // Clear central sudog cache. 2136 // Leave per-P caches alone, they have strictly bounded size. 2137 // Disconnect cached list before dropping it on the floor, 2138 // so that a dangling ref to one entry does not pin all of them. 2139 lock(&sched.sudoglock) 2140 var sg, sgnext *sudog 2141 for sg = sched.sudogcache; sg != nil; sg = sgnext { 2142 sgnext = sg.next 2143 sg.next = nil 2144 } 2145 sched.sudogcache = nil 2146 unlock(&sched.sudoglock) 2147 2148 // Clear central defer pool. 2149 // Leave per-P pools alone, they have strictly bounded size. 2150 lock(&sched.deferlock) 2151 // disconnect cached list before dropping it on the floor, 2152 // so that a dangling ref to one entry does not pin all of them. 2153 var d, dlink *_defer 2154 for d = sched.deferpool; d != nil; d = dlink { 2155 dlink = d.link 2156 d.link = nil 2157 } 2158 sched.deferpool = nil 2159 unlock(&sched.deferlock) 2160 } 2161 2162 // Timing 2163 2164 // itoaDiv formats val/(10**dec) into buf. 2165 func itoaDiv(buf []byte, val uint64, dec int) []byte { 2166 i := len(buf) - 1 2167 idec := i - dec 2168 for val >= 10 || i >= idec { 2169 buf[i] = byte(val%10 + '0') 2170 i-- 2171 if i == idec { 2172 buf[i] = '.' 2173 i-- 2174 } 2175 val /= 10 2176 } 2177 buf[i] = byte(val + '0') 2178 return buf[i:] 2179 } 2180 2181 // fmtNSAsMS nicely formats ns nanoseconds as milliseconds. 2182 func fmtNSAsMS(buf []byte, ns uint64) []byte { 2183 if ns >= 10e6 { 2184 // Format as whole milliseconds. 2185 return itoaDiv(buf, ns/1e6, 0) 2186 } 2187 // Format two digits of precision, with at most three decimal places. 2188 x := ns / 1e3 2189 if x == 0 { 2190 buf[0] = '0' 2191 return buf[:1] 2192 } 2193 dec := 3 2194 for x >= 100 { 2195 x /= 10 2196 dec-- 2197 } 2198 return itoaDiv(buf, x, dec) 2199 } 2200 2201 // Helpers for testing GC. 2202 2203 // gcTestMoveStackOnNextCall causes the stack to be moved on a call 2204 // immediately following the call to this. It may not work correctly 2205 // if any other work appears after this call (such as returning). 2206 // Typically the following call should be marked go:noinline so it 2207 // performs a stack check. 2208 // 2209 // In rare cases this may not cause the stack to move, specifically if 2210 // there's a preemption between this call and the next. 2211 func gcTestMoveStackOnNextCall() { 2212 gp := getg() 2213 gp.stackguard0 = stackForceMove 2214 } 2215 2216 // gcTestIsReachable performs a GC and returns a bit set where bit i 2217 // is set if ptrs[i] is reachable. 2218 func gcTestIsReachable(ptrs ...unsafe.Pointer) (mask uint64) { 2219 // This takes the pointers as unsafe.Pointers in order to keep 2220 // them live long enough for us to attach specials. After 2221 // that, we drop our references to them. 2222 2223 if len(ptrs) > 64 { 2224 panic("too many pointers for uint64 mask") 2225 } 2226 2227 // Block GC while we attach specials and drop our references 2228 // to ptrs. Otherwise, if a GC is in progress, it could mark 2229 // them reachable via this function before we have a chance to 2230 // drop them. 2231 semacquire(&gcsema) 2232 2233 // Create reachability specials for ptrs. 2234 specials := make([]*specialReachable, len(ptrs)) 2235 for i, p := range ptrs { 2236 lock(&mheap_.speciallock) 2237 s := (*specialReachable)(mheap_.specialReachableAlloc.alloc()) 2238 unlock(&mheap_.speciallock) 2239 s.special.kind = _KindSpecialReachable 2240 if !addspecial(p, &s.special, false) { 2241 throw("already have a reachable special (duplicate pointer?)") 2242 } 2243 specials[i] = s 2244 // Make sure we don't retain ptrs. 2245 ptrs[i] = nil 2246 } 2247 2248 semrelease(&gcsema) 2249 2250 // Force a full GC and sweep. 2251 GC() 2252 2253 // Process specials. 2254 for i, s := range specials { 2255 if !s.done { 2256 printlock() 2257 println("runtime: object", i, "was not swept") 2258 throw("IsReachable failed") 2259 } 2260 if s.reachable { 2261 mask |= 1 << i 2262 } 2263 lock(&mheap_.speciallock) 2264 mheap_.specialReachableAlloc.free(unsafe.Pointer(s)) 2265 unlock(&mheap_.speciallock) 2266 } 2267 2268 return mask 2269 } 2270 2271 // gcTestPointerClass returns the category of what p points to, one of: 2272 // "heap", "stack", "data", "bss", "other". This is useful for checking 2273 // that a test is doing what it's intended to do. 2274 // 2275 // This is nosplit simply to avoid extra pointer shuffling that may 2276 // complicate a test. 2277 // 2278 //go:nosplit 2279 func gcTestPointerClass(p unsafe.Pointer) string { 2280 p2 := uintptr(noescape(p)) 2281 gp := getg() 2282 if gp.stack.lo <= p2 && p2 < gp.stack.hi { 2283 return "stack" 2284 } 2285 if base, _, _ := findObject(p2, 0, 0); base != 0 { 2286 return "heap" 2287 } 2288 for _, datap := range activeModules() { 2289 if datap.data <= p2 && p2 < datap.edata || datap.noptrdata <= p2 && p2 < datap.enoptrdata { 2290 return "data" 2291 } 2292 if datap.bss <= p2 && p2 < datap.ebss || datap.noptrbss <= p2 && p2 <= datap.enoptrbss { 2293 return "bss" 2294 } 2295 } 2296 KeepAlive(p) 2297 return "other" 2298 } 2299