In-Reply-To: <20260116145208.87445-1-frederic@kernel.org> References: <20260116145208.87445-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-s390@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-DKIM: signer='kernel.org' status='pass' reason='' DKIMCheck: Server passes DKIM test, 0 Spam score X-Spam-Score: -1.2 (-) X-Spam-Report: Spam detection software, running on the system "witcher.mxrouting.net", has performed the tests listed below against this email. Information: https://mxroutedocs.com/directadmin/spamfilters/ --- Content analysis details: (-1.2 points) --- pts rule name description ---- ---------------------- ----------------------------------------- 0.0 RCVD_IN_DNSWL_BLOCKED RBL: ADMINISTRATOR NOTICE: The query to DNSWL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#DnsBlocklists-dnsbl-block for more information. [172.105.105.114 listed in list.dnswl.org] -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager -0.0 DKIMWL_WL_HIGH DKIMwl.org - High trust sender SpamTally: Final spam score: -11 The dyntick-idle cputime accounting always assumes that IRQ time accounting is enabled and consequently stops elapsing the idle time during dyntick-idle IRQs. This doesn't mix up well with disabled IRQ time accounting because then idle IRQs become a cputime blind-spot. Also this feature is disabled on most configurations and the overhead of pausing dyntick-idle accounting while in idle IRQs could then be avoided. Fix the situation with conditionally pausing dyntick-idle accounting during idle IRQs only if neither native vtime (which does IRQ time accounting) nor generic IRQ time accounting are enabled. Also make sure that the accumulated IRQ time is not accidentally substracted from later accounting. Signed-off-by: Frederic Weisbecker --- kernel/sched/cputime.c | 24 +++++++++++++++++++++--- kernel/sched/sched.h | 1 + 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index f0620b429698..3dadfaa92b27 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -45,7 +45,8 @@ static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, u64_stats_update_begin(&irqtime->sync); cpustat[idx] += delta; irqtime->total += delta; - irqtime->tick_delta += delta; + if (!irqtime->idle_dyntick) + irqtime->tick_delta += delta; u64_stats_update_end(&irqtime->sync); } @@ -80,6 +81,16 @@ void irqtime_account_irq(struct task_struct *curr, unsigned int offset) irqtime_account_delta(irqtime, delta, CPUTIME_SOFTIRQ); } +static inline void irqtime_dyntick_start(void) +{ + __this_cpu_write(cpu_irqtime.idle_dyntick, true); +} + +static inline void irqtime_dyntick_stop(void) +{ + __this_cpu_write(cpu_irqtime.idle_dyntick, false); +} + static u64 irqtime_tick_accounted(u64 maxtime) { struct irqtime *irqtime = this_cpu_ptr(&cpu_irqtime); @@ -93,6 +104,9 @@ static u64 irqtime_tick_accounted(u64 maxtime) #else /* !CONFIG_IRQ_TIME_ACCOUNTING: */ +static inline void irqtime_dyntick_start(void) { } +static inline void irqtime_dyntick_stop(void) { } + static u64 irqtime_tick_accounted(u64 dummy) { return 0; @@ -443,6 +457,7 @@ void kcpustat_dyntick_stop(ktime_t now) WARN_ON_ONCE(!kc->idle_dyntick); kcpustat_idle_stop(kc, now); kc->idle_dyntick = false; + irqtime_dyntick_stop(); vtime_dyntick_stop(); steal_account_process_time(ULONG_MAX); } @@ -454,6 +469,7 @@ void kcpustat_dyntick_start(ktime_t now) if (!vtime_generic_enabled_this_cpu()) { vtime_dyntick_start(); + irqtime_dyntick_start(); kc->idle_dyntick = true; kcpustat_idle_start(kc, now); } @@ -463,7 +479,8 @@ void kcpustat_irq_enter(ktime_t now) { struct kernel_cpustat *kc = kcpustat_this_cpu; - if (!vtime_generic_enabled_this_cpu()) + if (!vtime_generic_enabled_this_cpu() && + (irqtime_enabled() || vtime_accounting_enabled_this_cpu())) kcpustat_idle_stop(kc, now); } @@ -471,7 +488,8 @@ void kcpustat_irq_exit(ktime_t now) { struct kernel_cpustat *kc = kcpustat_this_cpu; - if (!vtime_generic_enabled_this_cpu()) + if (!vtime_generic_enabled_this_cpu() && + (irqtime_enabled() || vtime_accounting_enabled_this_cpu())) kcpustat_idle_start(kc, now); } diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index d30cca6870f5..cf677ff12b10 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -3307,6 +3307,7 @@ static inline void sched_core_tick(struct rq *rq) { } #ifdef CONFIG_IRQ_TIME_ACCOUNTING struct irqtime { + bool idle_dyntick; u64 total; u64 tick_delta; u64 irq_start_time; -- 2.51.1 From - Fri Jan 16 15:07:59 2026 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 Return-Path: Delivered-To: hi@josie.lol Received: from witcher.mxrouting.net by witcher.mxrouting.net with LMTP id EO5eEUtUamnvfDcAYBR5ng (envelope-from ) for ; Fri, 16 Jan 2026 15:07:55 +0000 Return-path: Envelope-to: hi@josie.lol Delivery-date: Fri, 16 Jan 2026 15:07:55 +0000 Received: from tor.lore.kernel.org ([172.105.105.114]) by witcher.mxrouting.net with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.98) (envelope-from ) id 1vglQZ-0000000FYcw-0FIP for hi@josie.lol; Fri, 16 Jan 2026 15:07:55 +0000 Received: from smtp.subspace.kernel.org (conduit.subspace.kernel.org [100.90.174.1]) by tor.lore.kernel.org (Postfix) with ESMTP id 977B93069012 for ; Fri, 16 Jan 2026 14:54:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9EA81394493; Fri, 16 Jan 2026 14:54:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="P58I4pON" X-Original-To: linux-s390@vger.kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B1833939B9; Fri, 16 Jan 2026 14:54:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768575250; cv=none; b=MdrcWGyUOLcDdu6d4KdglETnztOseyJfDN3gP5sYw+mcZKO0Q9llPno40e9h0ZxR73JEx45BPvRXAJLUkanUxPAUoHY3WxiblIE7gCDSJ8QSyVr3j4B2di1ZzkLqeRx1qfpwdnw5t1tk8qkkD3Q7HK9nwQkW9Qgjmpuy8nSSpW8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768575250; c=relaxed/simple; bh=NiFNP1dNEv/R0O+iuBotsMn8keb9Y+JzQXP3O6DHJSw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sVlpf1DtfKJ3mqV7Z1DQI47GgGOEwqjnpmv4j67EE9vVjWRJWmwd+EhteHnjtvrAowkCUuJPExnzQQMVTUC71Wmuvms08KBdI9GHa5HhsmQv8LthF2M59nzjsQDzaBa6Y8JYr2w9F8KNlSHXb1TaMTJbpFXPWswTVyxrQeA4650= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=P58I4pON; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4568DC19423; Fri, 16 Jan 2026 14:54:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1768575250; bh=NiFNP1dNEv/R0O+iuBotsMn8keb9Y+JzQXP3O6DHJSw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P58I4pONEqgRKgQ76ZkWUDt5XWzZMMI1w+pz+lcYFsRcbcsZHQupqMSvLSprbGueN b+pk/yo1pIdVpFUg3JXsmxcb/PGfRQU9pNdfd8Jr9hk0A1q6noGtXzxgWyURE9ouC3 1APMRpVzRm8n/zG3SHUGHD64MLVTPwa7TIMgoVeMS6BoBYJPpxS8Mw6kww+qUJDUgN uiIDVVVHWgH05pom4hl7O6l7IccpprH56k6WxOcl/M06cBnf3pySTKttXVjeZktmJd jPd5Cm2EqpHdhEMCmACSzFve7xNaBHvoUtLunX2kvWz82JJJyBOVA8xHbLW4Ftq6Hh ouLpAwlTIoQ0g== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , "Christophe Leroy (CS GROUP)" , "Rafael J. Wysocki" , Alexander Gordeev , Anna-Maria Behnsen , Ben Segall , Boqun Feng , Christian Borntraeger , Dietmar Eggemann , Heiko Carstens , Ingo Molnar , Jan Kiszka , Joel Fernandes , Juri Lelli , Kieran Bingham , Madhavan Srinivasan , Mel Gorman , Michael Ellerman , Neeraj Upadhyay , Nicholas Piggin , "Paul E . McKenney" , Peter Zijlstra , Steven Rostedt , Sven Schnelle , Thomas Gleixner , Uladzislau Rezki , Valentin Schneider , Vasily Gorbik , Vincent Guittot , Viresh Kumar , Xin Zhao , linux-pm@vger.kernel.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH 15/15] sched/cputime: Handle dyntick-idle steal time correctly Date: Fri, 16 Jan 2026 15:52:08 +0100 Message-ID: <20260116145208.87445-16-frederic@kernel.org> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20260116145208.87445-1-frederic@kernel.org> References: <20260116145208.87445-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-s390@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-DKIM: signer='kernel.org' status='pass' reason='' DKIMCheck: Server passes DKIM test, 0 Spam score X-Spam-Score: -1.2 (-) X-Spam-Report: Spam detection software, running on the system "witcher.mxrouting.net", has performed the tests listed below against this email. Information: https://mxroutedocs.com/directadmin/spamfilters/ --- Content analysis details: (-1.2 points) --- pts rule name description ---- ---------------------- ----------------------------------------- 0.0 RCVD_IN_DNSWL_BLOCKED RBL: ADMINISTRATOR NOTICE: The query to DNSWL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#DnsBlocklists-dnsbl-block for more information. [172.105.105.114 listed in list.dnswl.org] -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager -0.0 DKIMWL_WL_HIGH DKIMwl.org - High trust sender SpamTally: Final spam score: -11 The dyntick-idle steal time is currently accounted when the tick restarts but the stolen idle time is not substracted from the idle time that was already accounted. This is to avoid observing the idle time going backward as the dyntick-idle cputime accessors can't reliably know in advance the stolen idle time. In order to maintain a forward progressing idle cputime while substracting idle steal time from it, keep track of the previously accounted idle stolen time and substract it from _later_ idle cputime accounting. Signed-off-by: Frederic Weisbecker --- include/linux/kernel_stat.h | 1 + kernel/sched/cputime.c | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index e59916477075..a5b5a25c3cc1 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -39,6 +39,7 @@ struct kernel_cpustat { bool idle_elapse; seqcount_t idle_sleeptime_seq; ktime_t idle_entrytime; + u64 idle_steal; #endif u64 cpustat[NR_STATS]; }; diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 3dadfaa92b27..749a6ed4d2fa 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -424,18 +424,25 @@ static void kcpustat_idle_stop(struct kernel_cpustat *kc, ktime_t now) { u64 *cpustat = kc->cpustat; ktime_t delta; + u64 steal, steal_delta; if (!kc->idle_elapse) return; delta = ktime_sub(now, kc->idle_entrytime); + steal = steal_account_process_time(delta); write_seqcount_begin(&kc->idle_sleeptime_seq); + steal_delta = min_t(u64, kc->idle_steal, delta); + delta -= steal_delta; + kc->idle_steal -= steal_delta; + if (nr_iowait_cpu(smp_processor_id()) > 0) cpustat[CPUTIME_IOWAIT] = ktime_add(cpustat[CPUTIME_IOWAIT], delta); else cpustat[CPUTIME_IDLE] = ktime_add(cpustat[CPUTIME_IDLE], delta); + kc->idle_steal += steal; kc->idle_entrytime = now; kc->idle_elapse = false; write_seqcount_end(&kc->idle_sleeptime_seq); @@ -459,7 +466,6 @@ void kcpustat_dyntick_stop(ktime_t now) kc->idle_dyntick = false; irqtime_dyntick_stop(); vtime_dyntick_stop(); - steal_account_process_time(ULONG_MAX); } } @@ -507,6 +513,7 @@ static u64 kcpustat_field_dyntick(int cpu, enum cpu_usage_stat idx, if (kc->idle_elapse && compute_delta) { ktime_t delta = ktime_sub(now, kc->idle_entrytime); + delta -= min_t(u64, kc->idle_steal, (u64)delta); idle = ktime_add(cpustat[idx], delta); } else { idle = cpustat[idx]; -- 2.51.1 From - Fri Jan 16 15:08:47 2026 X-Mozilla-Status: 0001 X-Mozilla-Status2: 00000000 Return-Path: Delivered-To: hi@josie.lol Received: from witcher.mxrouting.net by witcher.mxrouting.net with LMTP id gJ4ZDntUamlWRTgAYBR5ng (envelope-from ) for ; Fri, 16 Jan 2026 15:08:43 +0000 Return-path: Envelope-to: hi@josie.lol Delivery-date: Fri, 16 Jan 2026 15:08:43 +0000 Received: from sto.lore.kernel.org ([172.232.135.74]) by witcher.mxrouting.net with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.98) (envelope-from ) id 1vglRK-0000000Fa7a-3dkn for hi@josie.lol; Fri, 16 Jan 2026 15:08:43 +0000 Received: from smtp.subspace.kernel.org (conduit.subspace.kernel.org [100.90.174.1]) by sto.lore.kernel.org (Postfix) with ESMTP id ABC0B3022CB0 for ; Fri, 16 Jan 2026 15:04:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [1