FreeBSD git weekly: 2026-07-01 to 2026-09-30

TLS receive offload is really only beneficial for in-kernel use cases
(such as NFS over TLS) or when using a hardware offload.  In addition,
several recent SAs have involved the TLS receive path, but the only
current mitigation for those is to disable TLS offload entirely.

Reviewed by:    ziaee, gallatin, markj
Relnotes:       yes
Sponsored by:   Netflix
Sponsored by:   Chelsio Communications
Co-authored-by: John Baldwin <jhb@FreeBSD.org>
Differential Revision:  https://reviews.freebsd.org/D57974

Start the loop by finding the end of the option name, the name-value
separator (if any), and the end of the option.  Use those pointers to
simplify matching the option name and parsing the option value, and
validate option names and values more strictly.  This means that:

* We no longer accept trailing garbage in an option name or value.  For
  instance, we would previously interpret “edns0123” as “edns0” and
  “timeout:3xyz” as “timeout:3”.  This was actually quite lucky because
  we also failed to recognize the newline at the end of the option line
  as a whitespace character.

* For options that take a numerical argument, we would previously accept
  negative values and treat non-numerical arguments as 0, while large
  numerical arguments would be capped to the option's maximum permitted
  value.  Now, any failure to parse the argument, including overflow,
  results in the option being left unchanged.

MFC after:      1 week
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57923

When we switched from the BIND4 resolver to the BIND9 resolver, the
sortlist parser was inadvertently disabled due to a missing #define, and
nobody seemed to notice.  The sorting code remained enabled in the
resolver, but there was no way to set a sort order.

Reimplement the sortlist parser, but correctly, and update the manual
accordingly.  The new parser accepts IPv4 and IPv6 addresses with or
without a mask or prefix length, just like the old one, except IPv6
support was a bit wonky in the original code.

Fixes:          https://cgit.freebsd.org/src/commit/?id=5342d17f09a8 ("Update the resolver in libc to BIND9's one.")
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57925

If a nhop gets an interface event, revalidate the nhops and
immediately try to recompile existing nexthop groups by
replacing unreachable nexthops with reachable ones.
If none are available, recompile them back to
their normal position in nexthop group slots.

Reviewed by:    glebius
Discussed with: markj
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D57389

The tcp_bblog facility provides structured logging of TCP stack activity
for debugging and performance analysis. It is implemented in the kernel
and allows per-connection tracing of TCP events with low overhead.

Reviewed by:    tuexen, ziaee
MFC after:      1 week
Relnotes:       yes
Differential Revision:  https://reviews.freebsd.org/D56252

Add the option "oemstring" to allow setting the DMI type 11 ("OEM
Strings") SMBIOS structure. These are free-form strings, available for
any purpose, but can be especially useful to pass configuration,
secrets, and credential information into a Linux guest and consumed by
systemd.

MFC after:      1 month
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57516

Point out which features are non-POSIX and thus can not be
safely assumed to be portable and exist in other implementations.

Relnotes:               YES!
Reviewed by:            ziaee, jilles
Differential Revision:  https://reviews.freebsd.org/D55333

Use pwait's new -r option to wait until the target processes have not
only terminated, but also been reaped.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=293183
MFC after:      1 week
Sponsored by:   Klara, Inc.
Sponsored by:   NetApp, Inc.
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D58391

Add the ability to select source ip address of outgoing packets
even when the source ip address is configured on another interface.
Also add this new rtnetlink attribute to manual.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=285422
Reviewed by:    glebius, ziaee (manpages)
Tested by:      ivy, Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D58294

Register the 82576 and I350 VF PCI IDs under a separate igbv driver
while continuing to share the igb datapath implementation.

Follow the ixv driver split and give the VF context IFLIB_IS_VF so
iflib does not apply the PF SR-IOV detach guard to a child VF. Program
VTIVAR_MISC in the VF low byte so mailbox and reset notifications reach
the VF admin vector.

The split will become increasingly obvious as bug fixes land, trying to
bias everything with if (sc->vf_ifp) everywhere is error prone in two
directions.

This breaks existing naming/configurations and cannot be MFCed as-is.
I have no plans of adapting it to prior branches at the moment but it
may be possible.

Relnotes:       yes
Sponsored by:   BBOX.io

Add the PCI IOV schema and PF control plane for up to seven VFs with
one hardware queue per pool. Implement VF mailbox handling, MAC and
VLAN assignment, multicast filtering, promiscuity policy,
anti-spoofing, malicious-driver recovery, reset replay, and queue
lifecycle management.

The basic SR-IOV and VMDq PF implementation follows DPDK Intel e1000
code, including PF pool selection, one queue per pool, mailbox
dispatch, and VF enablement. Intel FreeBSD igb-2.5.31 supplies the
older driver baseline. Linux igb and the Intel SDMs clear up lifecycle,
isolation, reset, and family-specific details absent from DPDK.

Enabling IOV requires the PF to attach with one TX and RX queue.
Systems whose defaults select RSS queues must set the documented iflib
queue override tunables before attach.

Only 82576 and I350 support SR-IOV in silicon.  The series has been
extensively tested on I350, including thowing boundaries at the PCI BAR
that shipping drivers will never.  Still, think carefully before
reaching for this in critical environments.

Relnotes:       yes
Sponsored by:   BBOX.io

Document supported controllers, PF and VF naming, PCI_IOV and IOMMU
requirements, queue and lifecycle constraints, iovctl schema, filtering
and anti-spoof policy, mailbox and MDD recovery, shared hardware
limits, rate control, and statistics cadence.

Relnotes:       yes
Sponsored by:   BBOX.io

Borrow the e1000 VLAN filter table

Ambiguous presence of the feature by Intel was settled by DPDK and
emperical testing.

MFC after:      2 weeks
Relnotes:       yes

The SR-IOV schema advertises MAC anti-spoofing and enables it by
default, but the VF configuration was never consumed and the hardware
policy remained disabled.

Record the configured policy and apply MAC and VLAN anti-spoofing
throughout VF initialization and reset. On X550-family devices, also
protect the LLDP and flow-control Ethertypes and enable per-VF
spoof-event accounting. Remove the driver-owned state during SR-IOV
teardown.

Adapt the anti-spoof configuration lifecycle used by igb(4) in
a2ed165f0049 to the ixgbe hardware controls.

MFC after:      1 week
Relnotes:       yes

The VF VLAN capability is checked but never granted, and no SR-IOV
configuration property exposes the existing default-VLAN support. PF
VLAN updates also replace VFTA registers from a PF-only shadow, erasing
live VF filters.

Expose access VLAN and trunk policy through the IOV schema. Track each
VF VLAN as desired state, restore the administrative VLAN after reset,
and use the native VLVF helper for incremental PF and VF ownership
changes.

Keep VLAN filtering enabled while SR-IOV is active. When PF hardware
filtering is disabled, admit every VLAN to the PF without bypassing
per-pool VF isolation. Reconstruct VLVF and the shared VFTA from PF and
VF desired state after reset or a filtering-mode transition, and
restore PF-only state on teardown.

When the last VF leaves a VLAN still owned by the PF, free its VLVF
slot while retaining the shared VFTA bit. This prevents a trunk VF from
exhausting the 64-entry VLVF table by cycling VLAN memberships.

Adapt the VLAN ownership model introduced for igb(4) in a2ed165f0049 to
ixgbe's native VLVF machinery.

Match Linux receive semantics by exposing a stripped VLAN tag only
when that VID was registered by the VF.  A PF-assigned port VLAN is an
administrative tag and must be delivered to the VF as untagged traffic;
otherwise the stack dispatches it to a nonexistent VLAN interface and
access-VLAN receive traffic is blackholed.

MFC after:      1 week
Relnotes:       yes

The allow-promisc IOV property is advertised but ignored, and the PF
rejects the xcast request used by modern VFs. Negotiate mailbox APIs
1.2 and 1.3, implement pool-scoped xcast modes, and require
allow-promisc for requested all-multicast or unicast-promiscuous modes.

The VF mailbox can carry only 30 multicast hashes. When ixv has a
larger list, request the API 1.2 all-multicast xcast mode instead of
extending the legacy SET_MULTICAST message. The PF grants that fallback
only to VFs configured with allow-promisc; otherwise ixv reports that
only the first 30 addresses are active.

Reset xcast state with the VF and have ixv replay the mode implied by
its interface flags after multicast updates.

Follow DPDK's ixgbe API 1.2/1.3 xcast contract, with allow-promisc
policy adapted from igb(4) in a2ed165f0049.

MFC after:      1 week
Relnotes:       yes

The PF advertises the legacy SET_MACVLAN mailbox request but always
rejects it. The request installs secondary unicast addresses.

Allocate an owned RAR pool for VF secondary addresses, reserve low
entries for PF filters, and place VF-primary addresses at the top of
the usable RAR range. Reject address collisions and cap each VF at
three secondary filters so one guest cannot exhaust the shared table.

Clear secondary filters on VF or PF reset and on SR-IOV teardown. This
hardware can anti-spoof only the VF primary source address. Reject
secondary filters while MAC anti-spoofing is configured, so installing
them requires an explicit administrative policy choice. Report optional
filter-table allocation failure without disabling SR-IOV.

Adapt the owned-RAR allocation and reset-cleanup model from igb(4) in
a2ed165f0049 to DPDK's ixgbe SET_MACVLAN mailbox semantics.

MFC after:      1 week
Relnotes:       yes

The shared X550 code provides malicious-driver detection, event
decoding, and per-pool recovery operations, but the PF never enables or
services them. A malformed VF descriptor can therefore go undetected
and avoid the per-pool recovery path supplied by the MAC.

Configure IOV state while VF DMA remains disabled, then enable MDD and
activate the VFs only after PF queue initialization is complete. On an
MDD event, withdraw mailbox CTS and gate the VF pool through PFVFTE and
PFVFRE. Retain the per-queue WQBR blocks until the VF enters a new reset
epoch; PFVFTE can still permit descriptor fetches into the internal
queue, so releasing WQBR early would allow a hostile VF to retrigger
MDD before it resets.

Send the non-CTS reset notification after servicing the VF mailbox.
Let a posted VF request win mailbox arbitration, defer notification if
the pass produced a response, and retry failed notifications from the
periodic admin pass. Poll WQBR so recovery does not depend on another
mailbox interrupt edge, while suppressing already-fenced pools. Latch a
PF reset request until the next hardware initialization.

The X550 datasheet defines every bit of WQBR_RX and WQBR_TX as a queue
bit, so an all-ones value is valid. Reject it only when IXGBE_STATUS,
which has reserved-zero bits, also reads as all ones and confirms dead
MMIO.

Temporarily disable MDD around live multiqueue SRRCTL drop-mode updates,
which hardware otherwise reports as queue-context changes. Serialize
that window with the iflib context lock and resample pending work after
MDD is restored.

Apply the per-pool recovery model used by igb(4) in a2ed165f0049 to the
existing DPDK-derived X550 hooks. The same register interface is
documented for X552 and X553, so cover the entire X550 family. Document
that VF traffic remains disabled until the reset handshake completes.

MFC after:      2 weeks
Relnotes:       yes

iflib counts resets initiated by its transmit watchdog in 69c3e0de01c1.

Export the counter in the per-device iflib sysctl tree so every
driver provides the diagnostic without a driver callback or duplicate
storage.

A watchdog reset does not establish how many packets failed.  It can
recover a hardware stall involving several queued packets or a missed
completion involving no packet loss.  Stop adding one output error per
watchdog event in em(4), igb(4), and igc(4).

Remove the redundant driver counters and move the diagnostic to
dev.<driver>.<unit>.iflib.tx_watchdog_events.

MFC after:      1 month
Relnotes:       yes

Add -L to query the generic packed-nvlist IOV_GET_STATUS interface.
Report PF enable state and configured and total VF counts.  For each VF,
print its PCI address, newbus attachment, bound driver, and ppt state.

Retry size negotiation if the topology changes between ioctls and reject
malformed or incompatible status records.

Keep NIC-specific operational state in ifconfig -v; iovctl owns the
device-neutral PCI topology and applies to any SR-IOV device class.

Relnotes:       yes

Add access and trunk VLAN policy to the SR-IOV schema.  Access VFs
use a hardware PVID and cannot alter their VLAN membership.  Trunk VFs
may register up to 16 VLANs, while VLAN 0 remains implicitly admitted
for untagged and priority-tagged traffic.

Enable hardware VLAN anti-spoofing and maintain the MAC-by-VLAN filter
cross-product used by DPDK.  Apply Linux's untrusted-VF limits of 18 MAC
addresses and 16 VLANs so one guest cannot consume the shared PF filter
table without bound.

Report the effective policy through the VF status interface and document
the iovctl schema.

MFC after:      2 weeks
Relnotes:       yes

10G-BX optics use paired wavelengths to carry 10 Gb/s Ethernet over a
single strand of single-mode fiber.  Their 10G compliance byte is
empty, so identify them from the SFF-8472 nominal signaling rate and
single-mode reach fields.

When an EEPROM also advertises 1G BASE-BX10, give the complete 10G
bitrate and reach signature precedence.  Otherwise retain FreeBSD's
permissive 1G-BX identification rather than requiring a nominal
1.3 GBd rate.

MFC after:      2 weeks
Relnotes:       yes

The bhyve_config(5) variable `virtio_msix` is namescoped to
`virtio.msix`. Configurations that have the old variable will
automatically be mapped to the new one, with a warning message printed
out.

Relnotes:       yes
Reviewed by:    ziaee, markj
Differential Revision:  https://reviews.freebsd.org/D58390

Reviewed by:  manu, adrian
Differential Revision:  https://reviews.freebsd.org/D58798

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296234(exp-run)
Relnotes:       yes
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D57772

E610 inherits the X550-family virtualization registers, anti-spoofing
controls, and malicious-driver operations, but the frontend does not
advertise SR-IOV and cannot negotiate the mailbox revision needed by
E610 VFs.

Initialize the X550-family PF/VF mailbox registers for E610 and use
PFVFLREC for its VF reset events, following DPDK shared ixgbe code.
Advertise the E610 SR-IOV capability, accept API 1.6 only on E610, carry
the existing xcast and queue operations forward to that revision, and
return the cached physical link speed and state with the three-dword
E610 operation.  Unsupported RSS and optional feature requests continue
to receive explicit failures.

SR-IOV activation also enables the existing X550-derived per-pool MDD
recovery path on E610.  Document the expanded protection and link-state
coverage.

Hardware validation created 63 VFs and rejected a 64th without flapping
the running PF.  Invalid TX and RX descriptor DMA independently asserted
the offender's WQBR bit, gated only that VF, preserved sibling traffic,
and recovered after the VF reset.  FreeBSD ixv, FreeBSD DPDK, Linux
ixgbevf, and Linux DPDK exercised the PF mailbox and data paths.

MFC after:      2 weeks
Relnotes:       yes
Sponsored by:   Dirk-Willem van Gulik from Web Weaving (E610 hardware)
Sponsored by:   BBOX.io

Unlike its GNU counterpart, our tail(1) has always errored out if given
repetitive or contradictory options, even prior to Keith Bostic's 1991
reimplementation.  There is no good reason to continue to do so, not
even tradition, since many other commands (including head(1)) simply
apply the rightmost option in cases like this.

MFC after:      1 week
Reviewed by:    allanjude, markj
Differential Revision:  https://reviews.freebsd.org/D58192

Now that fetchTimeout works reliably, setting an alarm is not only no
longer necessary but counterproductive, as it will trigger even if the
connection is not actually stalled but merely slow.

While here, improve the wording of the manual page's description of the
various options for setting a timeout.

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57911

While preparing GPT-schemed RaspberryPi images for the NanoBSD
Reimagined GSoC 2026 project, a discrepancy was identified between
mkimg(1) and gpart(8) regarding Microsoft Basic Data partitions (GUID
!ebd0a0a2-b9e5-4433-87c0-68b6b72699c7).

Currently, mkimg(1) relies on the MBR-centric name "ntfs" to identify
this partition type under the GPT scheme.  Conversely, gpart(8)
identifies this type as "ms-basic-data".

To allow automation scripts (such as those consuming from gpart backup)
to use a common partition type across tools, add ALIAS_MS_BASIC_DATA as
a valid alias.

This is part of a larger effort to avoid a custom, MBR-based image
generation logic for embedded SoCs like the Raspberry Pi, standardizing
on GPT layouts across all supported FreeBSD embedded devices.

Reviewed by:    imp
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58198

This allows to use output of '/usr/bin/time -ao foo' as direct input to
ministat(1).

While here make diagnostic message more verbose.

GNU hexdump supports octal and hex, we add supports for BSD style
hexdump for better compatibility.

See: https://github.com/llvm/llvm-project/pull/206581/

MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58074

- gnum4.c: fix m4_warnx() to use vwarnx() instead of warnx()
- eval.c: improve error messages for empty macro names
- extern.h: remove compute_prevep() declaration
- Update OpenBSD version strings

MFC After: 3 days

This partially reverts commit 77a201b1705dbd97ea9ebe5b25b1d4ddac8a7d38.

Requested by:   des, fuz

The fallback glyph is stored at index 0, and does not need to be
inserted into a mapping.

Previously there was a dead store of add_glyph's return value for the
fallback case, which upset Clang's static analyzer.  Now, cast the
return value to (void) to make it clear this is intentional.

Also change add_glyph's fallback parameter to a c99 bool to make its use
more clear.

Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D57174

install: Allow installing stdin

If from_name is "/dev/stdin" or "-" and the target is not a directory,
skip the comparison and copy data from standard input to the target.

MFC after:      1 week
Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D58348

install: Fix typo

MFC after:      1 week
Reported by:    markj
Fixes:          https://cgit.freebsd.org/src/commit/?id=d34870708db9 ("install: Allow installing stdin")

This is mainy focused on using bool for booleans but also renames some
variables for clarity, adds some explicit comparisons, adds some braces,
with miscellanous style fixes thrown in.

MFC after:      1 week
Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D58355

Check the `fdopen` return value before calling `cook_cat`.

Reviewed by:    markj, bnovkov
Differential Revision:  https://reviews.freebsd.org/D57741
MFC after:      1 week

pwait: Optionally wait until process is reaped

If the new -r option is specified, wait until the target process not
only terminates but is reaped.

MFC after:      1 week
Sponsored by:   Klara, Inc.
Sponsored by:   NetApp, Inc.
Reviewed by:    kib, markj
Differential Revision:  https://reviews.freebsd.org/D58314

pwait: Add a SIGINFO handler

On SIGINFO, print a space-separated list or remaining processes to
standard error.

MFC after:      1 week
Sponsored by:   Klara, Inc.
Sponsored by:   NetApp, Inc.
Reviewed by:    kib, markj
Differential Revision:  https://reviews.freebsd.org/D58386

rc.subr: Fix premature return from wait_for_pids

Use pwait's new -r option to wait until the target processes have not
only terminated, but also been reaped.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=293183
MFC after:      1 week
Sponsored by:   Klara, Inc.
Sponsored by:   NetApp, Inc.
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D58391

Bump dates

Fixes:          https://cgit.freebsd.org/src/commit/?id=c8f5e6819d4d ("pwait: Optionally wait until process is reaped")
Fixes:          https://cgit.freebsd.org/src/commit/?id=eddd8aa99ca8 ("pwait: Add a SIGINFO handler")
Fixes:          https://cgit.freebsd.org/src/commit/?id=356d0b79cf6f ("rc.subr: Fix premature return from wait_for_pids")

Sponsored by: AFRL, DARPA

Several functions were using sprintf() to write RPC server-controlled
data to a stack buffer.  Adopt some minimal changes from NetBSD to avoid
the potential overflows.

Security:       CVE-2026-16277
Security:       CVE-2026-16461
Reviewed by:    khorben
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58441

Reviewed by:  emaste
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58442

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks rpcgen during
the bootstrap build, since it assumes the return value is always
a mutable pointer.

For mkfile_output(), the pointed-to value is never modified, so
fix this by making the pointer const as well.

For open_log_file(), the current code modifies the supposedly const
value in-place to remove the filename suffix, which happens to work
but is wrong even in older versions of C.  Change the code to use a
printf "%.*s" format specifier to strip the suffix instead.

MFC after:      1 week
Reviewed by:    brooks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58489

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks sort during
the bootstrap build, since it assumes the return value is always
a mutable pointer.

As the returned pointer is never used to modify the value, fix this
by making the temporary variable const.

MFC after:      1 week
Reviewed by:    markj
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58491

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks xinstall during
the bootstrap build, since it assumes the return value is always
a mutable pointer.

As the returned pointer is never used to modify the value, fix this
by making the temporary variable const.

MFC after:      1 week
Reviewed by:    ray, markj, emaste
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58492

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks mkimg during
the bootstrap build, since it assumes the return value is always
a mutable pointer.

Make the existing 'sep' pointer const to fix the first case, and
for the second, introduce a new non-const pointer for strchr,
since we do modify the result in that case.

MFC after:      1 week
Reviewed by:    markj
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58493

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks m4 during the
bootstrap build, since it assumes the return value is always a
mutable pointer.

Since the returned value is never modified, simply make the
temporary const.

MFC after:      1 week
Reviewed by:    bapt, dim
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58494

Add some minimal handling of category sources other than static kernel
sources.  We don't actually look up dynamic sources yet (that would
require extended trace records to add the file names to the trace file
since we can't assume the trace file is running on a kernel with the
same numbers.)

Make the decision to append a "src/" prefix to each file name
dependent on the category source.

Reviewed by:    kib
Sponsored by:   Innovate UK
Differential Revision:  https://reviews.freebsd.org/D58412

* On SIGINFO, print the current path to stderr rather than stdout.

* Do so immediately, instead of the next time we finish a directory.

* Document this behavior in the manual page.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296861
MFC after:      1 week
Fixes:          https://cgit.freebsd.org/src/commit/?id=d1588599c024 ("Report the next directory being scanned ...")
Reviewed by:    wollman
Differential Revision:  https://reviews.freebsd.org/D58702

Reviewed by:  fuz, ngie
Co-authored-by: Robert Clausecker <fuz@FreeBSD.org>
Differential Revision:  https://reviews.freebsd.org/D58727

The current default has been unchanged for 14 years.  Increase it to
keep pace with modern hardware and software.  security/pinentry-gnome,
in particular, can sometimes need 112 kB.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297452
MFC after:      2 weeks
Sponsored by:   ConnectWise
Reviewed by:    cye, emaste
Differential Revision: https://reviews.freebsd.org/D58811

Our buffer is half a megabyte, but we are only initializing the first
two bytes.  Switching from static to dynamic initialization moves it
from .data to .bss, greatly reducing the size of the binary.

Fixes:          https://cgit.freebsd.org/src/commit/?id=cf74b63d61b4 ("yes: Completely overengineer")
MFC after:      1 week
Sponsored by:   Klara, Inc.
Reviewed by:    kevans
Differential Revision:  https://reviews.freebsd.org/D58890

While here, remove the long-unused dash in the first line.

Reviewed by:    ziaee, olce
Fixes:          https://cgit.freebsd.org/src/commit/?id=ddf144a04b53 ("ps.1: Revamp: Explain general principles, update to match reality")
MFC after:      1 day
Differential Revision:  https://reviews.freebsd.org/D58038

* Modernize the markup

* Describe the comment syntax

* Drop obsolete advice

* Capitalize sentences

* Improve the language

* Replace no_tld_query with no-tld-query; both are supported, but all the
  other multi-word options use hyphens rather than underscores.

* Add missing ENVIRONMENT section

* Redo the example

MFC after:      1 week
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57921

MFC after:    1 week

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D57124

Also do not start a new list for each flag item.

Reviewed by:    markj
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential revision:  https://reviews.freebsd.org/D57124

Submitted by: des
MFC after:      1 week

Submitted by: des
MFC after:      1 week

Reviewed by:  mckusick
Discussed with: markj
Tested by:      pho
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D57658

Commit 74654ba3b1b3 added and new chflags(1) flag called
"udontcache" or "dontcache".  This patch documents this flag.

This is a content change.

Reviewed by:    kib
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58181

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58123

Document the global fetchTimeout variable, now that it works reliably.

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57910

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58247

According to RFC 1918, the following IP prefixes are reserved for
private internets:

    10.0.0.0/8
    172.16.0.0/12
    192.168.0.0/16

This PR fixes the prefix lengths in references to private networks
("RFC 1918 networks", "the standard private IP address ranges").
The changes are limited to man pages.

Signed-off-by:  Yusuke Ichiki <public@yusuke.pub>
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2328
MFC after:      3 days

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58292

boot.9 was moved to kern_reboot.9, but this reference was not changed
appropriately.

Reviewed by:    mhorne, kib, emaste
Fixes:  https://cgit.freebsd.org/src/commit/?id=800e74955d4e ("boot(9): update to match reality")
MFC after:      3 days
Differential Revision:  https://reviews.freebsd.org/D58350

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58315

Point out which features are non-POSIX and thus can not be
safely assumed to be portable and exist in other implementations.

Relnotes:               YES!
Reviewed by:            ziaee, jilles
Differential Revision:  https://reviews.freebsd.org/D55333

List every AQC part aq_vendor_info_array[] probes, each with the
maximum speed aq_hw_capabilities() grants it.

Only the Atlantic 2 parts link at 10 Megabit.  The AQC100 and AQC100S
are the only SFP+ controllers; the rest are twisted pair.

Reviewed by:            adrian, ziaee
Signed-off-by:          Nick Price <nick@spun.io>
Differential Revision:  https://reviews.freebsd.org/D58144

Reviewed by:  ziaee, imp
Differential Revision:  https://reviews.freebsd.org/D58267

contigmalloc.9: Note that M_WAITOK may still return NULL

Reviewed by:    markj, bapt
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58382

contigmalloc.9: Correct typo

Reported by:    alc, rlibby
Fixes: https://cgit.freebsd.org/src/commit/?id=caabdb3aefdc ("contigmalloc.9: Note that M_WAITOK may still return NULL")

The manual page claimed that SIGINFO caused information to be printed to
stdout, when in fact it is printed to stderr, as one would expect.  This
has been true ever since the feature was first added in 2003.

MFC after:      1 week
Fixes:          https://cgit.freebsd.org/src/commit/?id=00d321a2b395 ("Add a SIGINFO handler.")
Reviewed by:    jilles
Differential Revision:  https://reviews.freebsd.org/D58392

Describe the disabled, adaptive, and low-latency settings and their
interrupt-rate tradeoffs.

MFC after:      1 week

Describe the disabled, adaptive, and low-latency settings and their
interrupt-rate tradeoffs.

MFC after:      1 week

Reviewed by:  markc
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58458

mknod.2: update the man page

State that FIFOs can be created, document the requirement that
dev must be zero then.  Mention whiteouts.

PR:     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297082
Reviewed by:    emaste
Sponsored by:   The FreeBSD Foundation
MFC after:      3 days
Differential revision:  https://reviews.freebsd.org/D58478

mknod.2: properly document root requirements

Submitted by:   Martijn Dekker <mcdutchie@hotmail.com>
PR:     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297082
Fixes:  https://cgit.freebsd.org/src/commit/?id=4090d103b0c3 ("mknod.2: update the man page")
MFC after:      3 days

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58463

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58463

There are expected to be additional improvements and this RELNOTES
entry will be updated accordingly.

Reviewed by:  des
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58483

Adjust the man page to what other LinuxKPI wlan man pages say and look
like as it has been a while since I wrote it.  The man page is not yet
hooked up to the build on purpose as the driver is not yet enabled in
the tree.

Sponsored by:   The FreeBSD Foundation
MFC after:      3 days
Reviewed by:    ziaee (earlier version)
Differential Revision: https://reviews.freebsd.org/D58479

There are few warnings reported by mandoc -Tlint:

bhyve_config.5:255:31: WARNING: new sentence, new line
bhyve_config.5:257:43: WARNING: new sentence, new line
bhyve_config.5:422:2: WARNING: missing section argument: Xr nm_open
bhyve_config.5:469:24: WARNING: skipping no-space macro
bhyve_config.5:483:2: WARNING: wrong number of cells: 2 columns, 4 cells
bhyve_config.5:484:2: WARNING: wrong number of cells: 2 columns, 4 cells
bhyve_config.5:541:24: WARNING: skipping no-space macro

 - "new sentence, new line" is a trivial formatting fix.
 - "missing section": there is actually no nm_open() manual page,
    so use .Nm instead of .Xr for it.
 - "no-space macro": format without .Oc and .Ns, similarly to
    how it is already done in bhyve.8 for VNC addresses.
 - "wrong number of cells": also a trivial fix.

Reviewed by:    jhb
Sponsored by:   The FreeBSD Foundation
MFC after:      3 days
Differential Revision:  https://reviews.freebsd.org/D58415

`nvmecontrol power -l ...` lists the available power modes.
Non-operational modes are marked with an asterisk. While here,
add <device-id | namespace-id> to the "nvmecontrol power" synopsis.

MFC after:              3 days
Reviewed by:            dab, imp, michaelo, ziaee
Differential Revision:  https://reviews.freebsd.org/D58480

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58264

Reviewed by:  emaste, mckusick
Sponsored by:   The FreeBSD Foundation
MFC after:      3 days
Differential revision:  https://reviews.freebsd.org/D58592

The IPv6 socket options IPV6_JOIN_GROUP and IPV6_LEAVE_GROUP
socket options are being extended to accept IPv4 multicast group
addresses in the RFC 3493 IPv4-mapped address format as a convenience
to application developers.

Caveat this addition carefully in the newly added HISTORY section,
addressing all previous review comments.

Approved by:    ziaee
Reviewed by:    ziaee, glebius
PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246
Differential Revision:  https://reviews.freebsd.org/D55382

The CTL High Availablity clustering feature allows a pair of hosts to
implement transparent failover.  The implementation uses a TCP
connection to exchange messages.  There is no authentication mechanism
and the protocol itself embeds kernel pointers in the messages exchanged
between HA hosts.  This property (of CTL_MSG_DATAMOVE messages
specifically), as well as insufficient validation of inbound messages,
mean that anyone able to access a CTL HA port is able to remotely
execute code on that host.

Provide a warning to this effect in the CTL man page.

Reported by:    Ryan of Calif.io
Reviewed by:    ziaee, ken, mav
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58622

Noted  and reviewed by:       lwhsu
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58666

Reviewed by:  ziaee
Differential Revision: https://reviews.freebsd.org/D58565

- Remove `\(em` from .Nm section as it's not valid mandoc markup.
- Remove the section from the .Nm directive (it's handled under the .Dt
  directive).

MFC after:      1 week
Reported by:    make manlint

Add missing commas after .Nm entries.

MFC after:      1 week
Reported by:    make manlint

MFC after:    1 week
Reported by:    make manlint

MFC after:    1 week
Reported by:    make manlint

The shared em(4) manual page lists only the em device-node name.
Document the /dev/led/igb* name as well.

MFC after:      2 weeks

Reviewed by:  emaste
Discussed with: imp
Fixes:  https://cgit.freebsd.org/src/commit/?id=802c6d5d61d1 ("cdefs.h: Introduce __nonstring attribute")
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58804

Reviewed by:  fuz
Approved by:    fuz (mentor)
MFC after:      1 month
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2288

MFC after:    3 days
Sponsored by:   fme AG

Document a few options that are currently supported but
not covered in bhyve_config(5):

 - monitor
 - vcpu.N.cpuset
 - domains.N.{size,cpus,domain_policy}
 - console (for arm64 and riscv)

MFC after:              1 week
Reviewed by:            bnovkov, jhb
Sponsored by:           The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58399

As far as I can tell, racct(2) has never existed, not even when
I added these references a decade ago.

Change them as commit e9e615c88a74 did in thr_new(2).

Reported by:    Karlo Miličević <karlo98.m@gmail.com>

Do not duplicate the documentation already available
through "sysctl -d", but tell the user where to find it.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257984
Suggested by:   Felix Johnson <felix.the.red@gmail.com>

The eXpat project has changed maintainers since this section was written
in 2002.  Update it to reflect reality.

Discussed with: Sebastian Pipping <sebastian@pipping.org>
Reviewed by:    bcr
MFC after:      3 days
Differential Revision:  https://reviews.freebsd.org/D58835

Standardize driver manuals on the style used for 12 years in vt(4).
This brings SYNOPSIS across all FreeBSD manual sections into harmony
of meaning where where SYNOPSIS lists available options, and does not
contain prose. Adjust mdoc(7) to reflect the established convention.

Reviewed by:    jhb
Discussed with: arch@ (marc.info/?l=freebsd-arch&m=176782215606871)
Differential Revision:  https://reviews.freebsd.org/D54586

asmc: try PIO before MMIO to avoid false T2 detection

Add hw.asmc.system-state and hw.asmc.board-id read-only sysctls to
expose the T2 system state register and Mac board identifier via SMC.

Try PIO access before MMIO during probe to prevent false T2 detection
on Macs that happen to have something mapped at the T2 BAR address.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57844

asmc: add system state and board identity sysctls

Add dev.asmc.0.system subtree with read-only sysctls for SMC diagnostic
and identity keys: shutdown_cause (MSSD), sleep_cause (MSSP),
thermal_status (MSAL), time_of_day (CLKT), power_state (MSPS),
board_id (RPlt), and chip_gen (RGEN).

Each sysctl is registered only if the key exists on the hardware.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57853

asmc: deduplicate sensor converters and cause sysctls

Replace per-type spXX_to_milli() functions with a table-driven
asmc_sensor_convert() that looks up the divisor by SMC type string.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57854

Book-E powerpc has 64-bit bus_addr_t but only a 32-bit bus_size_t.  Use
the right macros for maxsize and maxsegsize to fix the build.

Fixes:  https://cgit.freebsd.org/src/commit/?id=4bf8ce037 ("if_rge: initial import of if_rge driver from OpenBSD.")
Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57794

Depend on clknode_if.h in the module Makefile, so that it gets
explicitly built for the module.  Also, reduce the #if guards to only
the new clock output code, and gate them on all powerpc, not just
powerpc64.

Fixes:  https://cgit.freebsd.org/src/commit/?id=6b77d34f ("HYM8563: Add support for clock output.")
Reviewed by:    mmel
Differential Revision:  https://reviews.freebsd.org/D57795

These were added during the DPAA driver rewrite, and should not have
gone in then.  Remove them.

Reviewed by:  mav
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58003

PR:           https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199101
MFC after:      1 week
Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D57929

MFC after:    1 week
Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D57930

Some firmware delivers the power or sleep button press that woke the
system as an ordinary button press (Notify 0x80) shortly after resume,
rather than as the wakeup notification (Notify 0x02) the ACPI
specification requires for a button that is also a wake source.

On affected machines (e.g. the Framework Laptop 12, Intel Raptor Lake-P)
the power button is a control-method device behind the embedded
controller.  The EC latches the key press that woke the system across the
sleep transition and flushes it through its normal _Qxx query path as
soon as it is reinitialized on resume.  The replayed press is
indistinguishable from a genuine one, so the kernel honors it as a fresh
suspend request and the machine suspends again immediately after waking;
it cannot be kept awake with the button.

The event cannot be filtered at its source: it arrives over the same EC
query path that also carries legitimate events (lid, AC, thermal,
battery), so suppressing the drain would lose real notifications.
Instead, record the time of resume and ignore a button-initiated suspend
that arrives within a short grace window of it.  The timestamp is taken
before DEVICE_RESUME() re-initializes the EC, so it is set before the
replay can be processed on the ACPICA notify taskqueue; otherwise the
replay can be evaluated before the timestamp is written and slip through.
Measured from that point, the replay lands at ~600 ms across many cycles
on a Framework Laptop 12, whereas a deliberate press cannot occur that
quickly -- it happens well after the display is back -- so a one-second
window separates the two without ignoring real presses for any
perceptible time.

Spec-compliant firmware reports the wake as Notify 0x02, which is handled
on a different path and never reaches this check, so there is no change
in behavior on such systems.

The replay window is a fixed compile-time constant rather than a tunable
on purpose: it tracks a hardware characteristic -- the EC's post-resume
replay latency -- not a user policy, so there is no value a user would
meaningfully choose.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296243
Reviewed by:    adrian, imp (earlier revision), olce
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D57712

Also, add a check in the attach method that a per-CPU structure is
provided by the bus.  This allows to remove such checks in multiple
functions.  The check cannot currently fail as all x86 CPU drivers
(ACPI, legacy) provide the CPU_IVAR_PCPU instance variable, but it is
safer to have it, especially as an example to other driver writers.

Event:          Halifax Hackathon 202606
Location:       Seat 36K in AC667, still waiting for a gate at Montréal-Trudeau
Sponsored by:   The FreeBSD Foundation

Also, add a check in the attach method that a per-CPU structure is
provided by the bus.  This allows to remove such checks in multiple
functions.  The check cannot currently fail as all x86 CPU drivers
(ACPI, legacy) provide the CPU_IVAR_PCPU instance variable, but it is
safer to have it, especially as an example to other driver writers.

Event:          Halifax Hackathon 202606
Location:       Seat 25A in AF0349, before leaving Montréal-Trudeau
Sponsored by:   The FreeBSD Foundation

This fixes associating to various APs.

It worked fine to a FreeBSD AP (which is a wholly separate problem
I'm going to need to dive into) but not to my tplink AX1800 Wifi-6
router.

PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296503

Locally tested:

* STA: Intel Centrino Advanced-N 6205 (iwn), Lenovo T420
* AP: TP-Link AX1800 wifi-6 router

M_PREPEND in the broadcast branch may call m_prepend(9) which allocates
a new head mbuf and calls m_move_pkthdr(), stripping M_PKTHDR from the
old mbuf.

xfer->mbuf was set before M_PREPEND, so it pointed at the
deheadered old mbuf. bus_dmamap_load_mbuf(9) asserts M_PKTHDR and
panics.

Reviewed by:    zlei, adrian
Differential Revision:  https://reviews.freebsd.org/D57495

Use kn->kn_sdata to track the last bs->total value for each knote
attached to an mmaped channel. An event is delivered only when the total
byte counter has advanced by at least c->lw since the last delivery.
After delivery kn_sdata is updated to the current total.

Each knote tracks its own watermark independently, so multiple knotes
attached to the same mmaped channel all receive events correctly.

Non-mmap channels keep the existing level-triggered behavior via
chn_polltrigger().

MFC after:      1 week
Reviewed by:    christos
Differential Revision:  https://reviews.freebsd.org/D57833

arm64/vmm: Add FEAT_NV2 definitions

Add the definitions for the VNCR_EL2 register and all of the offsets
to registers in memory relative to the page stored in VNCR_EL2.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56550

arm64/vmm: Use the VNCR_EL2 memory page to store guest registers

Wherever possible, move the storage space for guest register values from
the hypctx struct into a preallocated memory page matching the layout of
the page pointed to by VNCR_EL2.
This will streamline implementing support for nested virtualization, but
the implementation itself is not reliant on the presence of nested
virtualization architecture features.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56551

arm64/vmm: Store non-VNCR registers in an array

Move non-VNCR EL0 and EL1 registers into a dedicated array inside of
hypctx. This enables uniform accesses to both VNCR and non-VNCR guest
register state through hypctx_[read|write]_sys_reg().

The accessors are _not_ used for non-VNCR EL2 registers in order to
create a clear separation between guest-visible and guest-invisible
register state.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56552

arm64/vmm: Refactor vmm_hyp.c

Refactor vmm_hyp.c to split register reload logic by type of register,
streamline the implementation and improve readability.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56553

arm64/vmm: Move vttbr_el2 & vtimer into struct hypctx

Move vttbr_el2 & vtimer from struct hyp into struct hypctx to streamline
the logic and handle them in the same way as other *_el2 registers are
already being handled.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56554

arm64/vmm: Move host-side EL2 regs into sys_regs

Move EL2 host registers that are not visible to the guest into
hypctx->sys_regs. Prefix them with HOST_ to distinguish from EL2
registers which are part of the guest's own state (e.g. in VNCR).

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56555

arm64/vmm: Make remaining registers use hypctx_*_sys_reg

Move vgic, timer and trapframe registers into sys_regs to handle them
in the same way as all the other registers.

Signed-off-by:  Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed by:    andrew
Sponsored by:   Arm Ltd
Differential Revision:  https://reviews.freebsd.org/D56556

Fix tid_set_busy() for when `pmap` is NULL. Obviously a NULL pointer
cannot be correctly used, so I'm not sure how it worked in testing on
64-bit.

When the TID rolls over on a given CPU, simply flash-invalidate the
TLB instead of walking the TLB to only invalidate the repurposed TID.
Walking 256 entries is expensive, and we'll likely be inserting a bunch
new ones anyway in the new environment, since 256 really only handles
1MB of storage, so the likelihood of other mappings continuing to exist
in the TLB when their thread owner is scheduled again is very very
small.

DEVX event notifier returned true for the command-completion and
page-request events. This is causing mlx5_eq_int() to skip the core EQ
handler, so the firmware command interface and the page supply stop
being serviced and the device wedges.

This commit also make notifier registration and dispatch safe against
the EQ interrupt running concurrently: publish the table pointer before
the callback and load it with acquire semantics. run the callback under
RCU, and drain it with synchronize_rcu() on teardown. Otherwise the
interrupt handler could observe a half-initialized notifier or race with
cleanup.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

Import Linux upstream commits a8b92ca1b0e5ce620e425e9d2f89ce44f1a82a82
and c59450c463695a016e823175bac421cff219935d.

The DEVX object and method definitions were already present, but nothing
pointed ib_device.driver_def at them.  ibcore therefore never merged
them into the uverbs uapi tree and every DEVX ioctl came back as
EPROTONOSUPPORT.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

Import Linux upstream commit 342ee59de98a2ecdf15a46849a2534e7c808eb1f.

The dynamic UAR object was declared in the ABI headers but had no handler,
so the ioctl was rejected and dynamic-UAR contexts could not allocate a
doorbell UAR at all.

Implement the alloc and destroy methods following the upstream driver:
grab a UAR stamped with the caller's DEVX uid, expose it to user space
through an rdma_user_mmap entry (write-combining or non-cached as
requested), and free it on destroy.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

A firmware object owned by a DEVX uid may only reference resources owned
by the same uid or ones explicitly marked as shared.  Completion EQs
were created with uid 0, so a CQ owned by a DEVX uid could not attach to
its EQ and CREATE_CQ failed with "bad resource".

Create completion EQs with MLX5_SHARED_RESOURCE_UID on devices that
support user contexts, so uid-owned CQs can use them.

The code follows the Linux commit d2c8a1554c10d5e0443b1f97f480d7dacd55cf55
("IB/mlx5: Enable UAR to have DevX UID").

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5ib: allocate IB queue counters as a shared resource

A QP owned by a DEVX uid references the port's queue counter.  The
counter was allocated with uid 0, so RST2INIT_QP on a uid-owned QP
failed with "bad resource state".

Allocate and free the IB queue counters directly and, on devices that
support user contexts, stamp them with MLX5_SHARED_RESOURCE_UID so
uid-owned QPs can use them.

The code follows the Linux commit d2c8a1554c10d5e0443b1f97f480d7dacd55cf55
("IB/mlx5: Enable UAR to have DevX UID").

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5ib: encode dynamic UAR mmap offsets in the reserved command range

The UAR ioctl handed user space a raw mmap offset, so the first dynamic
UAR landed at page offset 0.  mlx5_ib_mmap() decodes offset 0 as the
legacy regular-page command and routed the mapping through the old bfreg
path, which rejects dynamic-UAR contexts, so mmap() failed with EINVAL
and mlx5dv_devx_alloc_uar() returned NULL.

Follow the upstream scheme: reserve the mmap command range [9, 255] for
rdma_user_mmap entries and return command-encoded offsets, so the
dynamic-UAR mappings decode to the intended mlx5_ib_mmap() path.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5ib: advertise write-combining support for dynamic BlueFlame UARs

Import Linux upstream commit 1f3db161881b7e21efb149e0ae8152b79a571a8f.

dev->wc_support was never set, so it was always false and the UAR ioctl
refused BlueFlame (write-combining) UAR allocations with EOPNOTSUPP.
That breaks QP creation in pure dynamic-UAR mode, where user space asks
for a BF doorbell UAR.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5: pass the full EQE to the DEVX event notifier

The DEVX event notifier and its helpers expect a full struct mlx5_eqe
and read eqe->data from it, but mlx5_eq_int() passed &eqe->data, so the
data offset was applied twice.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5: guard against a NULL CQ event handler in mlx5_cq_event()

DEVX and mlx5en created CQs are registered without an asynchronous
event handler (mcq.event is NULL).  An asynchronous CQ_ERROR event for
such a CQ made mlx5_cq_event() call through a NULL pointer and panic.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

mlx5: propagate the DEVX uid through SRQ create and destroy

The SRQ command builders never stamped the owning DEVX uid into the
firmware CREATE_SRQ/CREATE_RMP/CREATE_XRC_SRQ commands, so a basic SRQ
was always created with uid 0.  Every modern libmlx5 context runs with a
DEVX uid, and the QPs that reference the SRQ carry that uid, so firmware
rejected CREATE_QP with "bad resource": a uid-owned QP may not reference
a uid-0 SRQ.

Reviewed by:    kib
Tested by:      Wafa Hamzah <wafah@nvidia.com>
Sponsored by:   Nvidia networking
MFC after:      1 month

The thermal interrupt is initially masked.

Thermal interrupt handling is enabled by calling lapic_enable_thermal(),
which installs a (single) handler.

[olce: Wrote the commit message.]

Reviewed by:    kib, olce
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D44454

acpi: Add a pseudo-bus for APEI devices to manage resources

Different APEI tables can reuse the same registers (and sometimes
different views of the same register, e.g. 32- vs 64-bit mappings of
the same register).  To enable this sharing, apei0 now acts as a bus
device managing a pool of allocated resources and handing out mappings
to child devices which handle individual tables.

Most of the previous apei(4) driver has been moved into a new
hest0 device that is a child of apei0.

Reviewed by:    gallatin
Sponsored by:   Netflix
Differential Revision:  https://reviews.freebsd.org/D58024

acpi: fix instant panic in hest_attach()

Since now there is a pseudo-bus between our device and acpi0, we need to
go deeper.

Fixes:  https://cgit.freebsd.org/src/commit/?id=9313f6b01485ad9a0b7cc59b459f5714533587c3

Fixed the post-LPS delay from 500us to the IEEE 1394a-2000 s6.1 mandated
10ms ceiling. Handled PHY_INT by clearing W1C status bits in register 5
(masked ISBR to avoid spurious bus resets). Added a SID timeout callout
that recovers the state machine when a remote device fails to complete
self-ID. Fixed FW_PHY_SPD operator precedence and gated noisy messages
behind bootverbose/firewire_debug.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58033

fwcam: add dynamic resolution and frame rate support

Read V_MODE_INQ and V_RATE_INQ registers for all supported formats
during probe, caching the camera's actual capabilities.  Use these
to validate SMODE ioctl requests before writing to the camera.

Writing an unsupported combination caused the camera to
stop responding, requiring a physical power cycle.

Tested with: Apple iSight (external FireWire)

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58090

fwcam: set ISO speed from device link speed

iso_speed was never initialized, defaulting to S100 regardless of the
camera's actual link speed. Some cameras firmwares reject ISO_EN
when the speed field in the ISO_CHANNEL register
does not match their capabilities.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58091

fwcam: write video mode registers before enabling ISO streaming

The IIDC spec (s3.1) requires the video mode to be programmed before ISO
enable. Without this, cameras that power up with invalid default
mode/rate combinations reject the ISO_EN write.

This can happen when the firmware of teh camera is outdated or
vendor never updated it.

Differential Revision:  https://reviews.freebsd.org/D58092

Sponsored by; The FreeBSD Foundation
MFC after:      1 week

Sponsored by: The FreeBSD Foundation
MFC after:      1 week

Moved ISO start to first usage. Opening the device now
only validates state and increments the open count, allowing info
queries and mode changes without starting the camera. ISO streaming
begins on demand when userland first reads frame data.
This avoid the camera led to turn-on at attach.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58100

Some IIDC cameras power down the sensor when inactive (e.g. lens
cover closed) and reject ISO enable with EIO. Re-power the camera
and retry once before failing.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58101

MFC after:    1 week
Sponsored by:   The FreeBSD Foundation

USB vendor:product 184f:0051

Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D56794

This can't be a loadable module, so add it to MINIMAL

Sponsored by:           Netflix
Differential Revision:  https://reviews.freebsd.org/D58067

This makes the code slightly more compact and easier to read.

No functional change intended.

Reviewed by:    bnovkov
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58110

The thermal LVT slot does not necessarily exist.

According to Intel's Software Developers Manual, for Intel processors
supporting 64-bit operation (amd64), probably even the earliest ones
should have a local APIC with such a slot (the slot was introduced with
Pentium 4 and Xeon processors according to the manual, and the 64-bit
implementation in some later versions of them).  AMD's Architecture
Programmer's Manual also seems to imply that all AMD processors
supporting amd64 should have the slot too.  So this change may not be
needed when i386's code is dropped, but it does not hurt to have it, and
it might ease possible MFCs.

Change the signature of lapic_enable_thermal() so that it can report
failure (if there is no local APIC or if there is no thermal LVT slot).

Reviewed by:    bnovkov, kib
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58086

Differential Revision:        https://reviews.freebsd.org/D56923
Reviewed by:    mhorne

This change implements the equivalent of the amd64-specific
'show pte' ddb command used to dump the page table entries
associated with a specific virtual address.

Differential Revision:  https://reviews.freebsd.org/D56924
Reviewed by:    mhorne

Spurious page faults caused by cached invalid entries may occur when
starting APs and potentially panic the kernel if we're running in
a non-sleepable context.

Fix this avoidable panic by flushing the TLB after the AP is released.

Differential Revision:  https://reviews.freebsd.org/D57003
Reviewed by:    markj

The Privileged ISA specification permits caching of invalid PTEs
12.2.1. Supervisor Memory-Management Fence Instruction), which may
result in a spurious page fault. Such faults are handled by 'pmap_fault'
which locks the kernel pmap before inspecting and possibly updating the
offending L2 entry.
Unfortunately, spurious faults may also occur when we're already holding
the kernel_pmap lock or running in a critical section, where any attempt
to grab the pmap lock will result in a kernel panic.

Fix this avoidable panic by performing a lockless lookup to determine
whether a valid kernel mapping exits and flushing appropriate TLB entry.

Differential Revision:  https://reviews.freebsd.org/D56925
Reviewed by:    jrtc27, mhorne, markj

Using cpu_get_pcpuid() directly or having a CPU ID cache does not really
make any significant difference.  With cache: Less function calls, less
space on stack, but an additional allocation in the softc, who stays
permanently.  Without cache: Some function calls, but one less slot in
the softc, and no data duplication (but that info never changes).

The main reason for this change is to reduce conflicts with some
work-in-progress by aokblast@.

While here, move the check that a per-CPU structure is provided by the
bus from the attach to the probe method, as it is already used by
hwpstate_probe_pstate() there.

Reviewed by:    aokblast
Sponsored by:   The FreeBSD Foundation

To minimize the diff with hwpstate_intel(4).  See previous commit there
for the rationale.

Reviewed by:    aokblast
Sponsored by:   The FreeBSD Foundation

KVM does not always use 0x40000000 as its CPUID base. For example, QEMU
adds a 0x100 offset when nested virtualization is detected and the host
exposes Hyper-V enlightenment hints. To accommodate this behavior,
switch the detection logic to use the CPUID leaf returned by do_cpuid(),
making the implementation more flexible.

See:
https://github.com/qemu/qemu/blob/master/target/i386/kvm/kvm.c#L2300

Reviewed by:    kib
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58146

Presumably surfaced by -fstack-protector-strong, rk8xx_settime was
triggering SSP when ntpd set the time on the RockPro64, at the very
least.  A minor oops meant that the weeks mask was getting tossed into
the wrong field, and the mask was never populated.  The mask is 0x7 for
all three of these, thus overflowing the `data` array in settime by one
byte.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296719
Reported by:    jsm, "Tenkawa" on Discord
Reviewed by:    mmel
Differential Revision:  https://reviews.freebsd.org/D58182

Using if_getflags() to check IFF_DRV_RUNNING is wrong;
if_getdrvflags() is required.  This issue resulted in the
multicast filter not being updated.

This was an oversight by me in my initial port.

Thanks to danilo@ for reporting it and Oleg <oleglelchuk@gmail.com>
for the fix.

PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295176

Currently, devd emits events for external adapters only.

Send Netgraph init/disconnect events to devd so the internal adapter's
state could be asserted from userland.

(adrian - indentation changes.)

Signed-off-by: Kirill Orlov (-k) <slowdive@me.com>
Reviewed-by: adrian, imp
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2196

Matches tcpdump naming, but without getting more intense as you add more
-t.  This slightly reduces the post-processing needed on usbdump output
to diff two transactions.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58196

Introduce fdt_ether_get_addr() in fdt_common.c/h that tries standard
DT properties in the correct order and falls back to a random address
when needed. This should be used by ethernet drivers instead of open-coding
the same logic.

MFC after:      2 weeks

Reviewed by:    mhorne, adrian, bz, jrtc27
Differential Revision:  https://reviews.freebsd.org/D58104

When compiled without 'options RSS', the ena driver created taskqueues
using taskqueue_start_threads_cpuset passing a mask value of NULL,
both in the ena_setup_tx_resources path (for enqueues) and in the
ena_create_io_queues path (for the completion-processing).

In the default configuration, on most EC2 instances, this results in
taskqueues running in the right NUMA domain, but only by accident; in
non-default configurations (e.g. with with multiple EBS volumes
attached and associated NVMe taskqueues) the taskqueues may land in
the wrong NUMA domain even on instance types where the one-EBS-one-ENA
case produces the desired results.

Set (struct ena_que)->domain and use that to inform the choice of CPU
sets.  On a c8gn.48xlarge EC2 instance this doubles throughput on a
32-TCP-stream benchmark.

Reviewed by:    akiyano
MFC after:      7 days
Sponsored by:   Amazon
Differential Revision:  https://reviews.freebsd.org/D57918

In the DEVX_SUBSCRIBE_EVENT handler the eventfd path can fail and
"goto err" before the subscription's xa keys and ev_file have been set;
they are still zeroed from kzalloc().  The cleanup then looks up a
level-1 xa entry with key 0, gets NULL, and faults dereferencing it.

Initialize the fields the cleanup path relies on right after the
subscription is allocated, before it is linked and before the fallible
fdget(), so a later failure unwinds cleanly.

Reviewed by: kib
Sponsored by: Nvidia networking
MFC after: 1 month

The DEVX_SUBSCRIBE_EVENT redirect path resolved the user's eventfd with
fdget(), which on FreeBSD only finds LinuxKPI files.  rdma-core creates
the eventfd with the native FreeBSD eventfd(2), so the lookup failed and
subscription returned EBADF; the delivery side likewise assumed a
LinuxKPI-pollable file.

Use the LinuxKPI eventfd_ctx API instead: eventfd_ctx_fdget() resolves
the native eventfd, eventfd_signal() notifies it, and eventfd_ctx_put()
releases it.  DEVX async events can then be delivered through a redirect
eventfd.

Reviewed by: kib
Sponsored by: Nvidia networking
MFC after: 1 month

Fixes:                https://cgit.freebsd.org/src/commit/?id=fc9dc8482396 ("snd_uaudio: Lock usbd_transfer_start() in uaudio_mixer_ctl_set()")
PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296682
Sponsored by:   The FreeBSD Foundation
MFC after:      3 days

virtio: Add feature bit definitions up to VirtIO v1.3

Signed-off-by: Faraz Vahedi <kfv@kfv.io>
Reviewed-by: ngie
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2319

virtio: Report feature masks on negotiation failure

Signed-off-by: Faraz Vahedi <kfv@kfv.io>
Reviewed-by: ngie
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2319

virtio: Accept VIRTIO_F_RING_RESET in the modern PCI transport

Accept per-virtqueue reset when the device offers it, alongside
the V1 flag.  Negotiating the feature merely permits the use of
per-virtqueue reset and imposes no obligation on a driver that
never uses it, while refusing capability-only transport features
can make strict devices reject the feature set altogether.

No functional change on hosts that do not offer RING_RESET.

Signed-off-by: Faraz Vahedi <kfv@kfv.io>
Reviewed-by: ngie
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2319

This is to prevent child drivers from using the features returned by
previous drivers (in an arbitrary order).  None of the existing ones do
that, so this is purely defensive.

MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation

This is needed for VM_PHYS_TO_PAGE() to work, which is needed for
pmap_map_io_transient() to work, which is needed for uiomove_fromphys()
to work.

PR:     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296348
Reported and tested by: Anton Saietskii <vsasjason@gmail.com>
Reviewed by:    markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58274

As RX processing is heavier than TX completions processing, swap the
order and process TX completions first, in order to avoid starving the
completions and causing potential missing TX completions.

Submitted by: Ofir Tabachnik <ofirt@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58239

Move per-packet counter_enter/counter_exit pairs out of the RX
processing loop and batch them into a single update after the
loop completes.

Previously, each received packet triggered two separate
counter_enter/counter_exit blocks -- one for bytes and one for
packet count. This commit accumulates totals in local variables
and updates all four counters (ring and hw stats for both packets
and bytes) in a single counter_enter/counter_exit block after the
loop.

Also move the stats update to after the refill and LRO flush
so that the error path (goto update_stats) and the normal path
converge at the same label, avoiding code duplication.

Submitted by: David Arinzon <darinzon@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58240

Sporadic 'Found a Tx that wasn't completed on time' warnings appear
under sustained TX load, always reporting '1 msecs since last cleanup'
despite the 5-second timeout threshold.

The per-packet TX timestamp uses struct bintime (128 bits: two 64-bit
fields sec and frac) which is read and written non-atomically. A race
exists between the missing TX completion check
(check_missing_comp_in_tx_queue reading the timestamp) and the TX
submit path or cleanup path writing it on another CPU. Since the two
fields are not updated atomically, the check can observe a partially
written timestamp - one field from the old value and one from the new.
This can produce a timestamp with {sec=0, frac=valid}, causing the
check to compute a time offset equal to system uptime and falsely
exceeding the 5-second timeout.

Confirmed by instrumentation showing all occurrences had sec=0 with
valid frac/mbuf, cleanup_running=0, and ticks==last_cleanup_ticks.

Replace struct bintime with sbintime_t (a single 64-bit value) for
tx_buf->timestamp. An aligned 64-bit store/load cannot be torn on
64-bit architectures. Additionally, snapshot the timestamp into a
local variable in the check path to prevent a read-then-read race
where the timestamp could be zeroed between the zero-check and the
offset calculation.

Testing:
On m6i.large (FreeBSD 15.0-RELEASE-p6 amd64, 2 IO queues), two
instances with MTU 1500. Ran iperf -P 20 -u -b 320kpps (CPU
saturated at ~7 Gbps aggregate).

Without the fix: 8 warnings in 6 hours (first at ~72 min).
With the fix: 0 warnings after 20+ hours under identical conditions.

Fixes: https://cgit.freebsd.org/src/commit/?id=9b8d05b8ac78 ("Add support for Amazon Elastic Network Adapter (ENA) NIC")
Submitted by: Gilad Ben Yakov <giladben@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58241

Bug Fixes:
* Fix false 'missing TX completions' warnings due to timestamp race
* Put taskqueues into correct NUMA domain if !RSS

Minor Changes:
* Batch RX statistics updates
* Swap RX/TX completions cleanup order

Submitted by: Arthur Kiyanovski <akiyano@amazon.com>
MFC after: 2 weeks
Sponsored by: Amazon, Inc.
Reviewed by: cperciva
Differential Revision: https://reviews.freebsd.org/D58242

Added structure to allow multiple device to attach to the same driver.
Also removed the deprecation warning from the man page.

Differential Revision:  https://reviews.freebsd.org/D58201
Reviewed by:    adrian

Migrated fwcam to use per-unit-directory child device

Differential Revision:  https://reviews.freebsd.org/D58202
Reviewed by:    adrian

Migrated fwisound to use per-unit-directory child device

Differential Revision:  https://reviews.freebsd.org/D58203
Reviewed by:    adrian

Migrated fwdv to use per-unit-directory child device

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58204

SPL is a no-op on amd64. Real locking is already handled by fc_mtx and
per-driver mutexes.

Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D58210

Otherwise we try to disable the wrong IRQ.

Fixes:          https://cgit.freebsd.org/src/commit/?id=47e073941f4e ("Import the kernel parts of bhyve/arm64")
MFC after:      1 week

Now that IRQs can properly be disabled by GICD_ICENABLERn, an EOI for a
disabled IRQ ends up being lost, since we don't assign it to a list
register and don't enable maintenance interrupts for such cases. As a
result, we keep the IRQ active, which stops it from ever being delivered
again (which would be true even if we supported the active and pending
state). Keep disabled but active IRQs around in list registers so we can
see the EOI having taken place in a future sync (noting that since we
already don't create list registers in active and pending state there
are no concerns with causing a disabled IRQ to be delivered).

Fixes:          https://cgit.freebsd.org/src/commit/?id=47e073941f4e ("Import the kernel parts of bhyve/arm64")
MFC after:      1 week

dpaa2_ni_init() only enabled the DPNI object; it never pushed the
promiscuous/allmulti state or the multicast filter table to the MC
firmware.  The SIOCSIFFLAGS handler ignores flag changes that arrive
while the interface is down, yet still latches them into sc->if_flags,
so a promiscuous mode request made before the first up was silently
lost and could never be applied afterwards: the up path runs
dpaa2_ni_init(), which did not read the flags, and every later
SIOCSIFFLAGS compares against the already-latched value and sees no
change.

This is exactly what happens when if_bridge adds a dpni member while
the dpni is still down, e.g. rc.conf's

    create_args_bridge0="... addm dpni0"

running at bridge clone time, before ifconfig_dpni0="up" is processed.
bridge_ioctl_add() puts the member into promiscuous mode at addm time;
the request never reaches the firmware, so the DPNI continues to
hardware-filter unicast destined to other MACs.  ifconfig still
reports PROMISC (a stack-level flag), which makes the failure
invisible: the host stays reachable only via the DPNI's own MAC
address (e.g. with net.link.bridge.inherit_mac=1), while bridged
epair/vnet jail traffic is silently dropped on RX.

Reapply both pieces of administrative state after enabling the DPNI,
as other NIC drivers do in their init path.  This also restores
multicast memberships joined while the interface was down.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=292006
Reported by:    jhibbits

Signed-off-by: Nick Price <nick@spun.io>

Reviewed by:    jhibbits
Differential Revision:  https://reviews.freebsd.org/D58330

This fixes a build break for i386.

Reviewed by:    kib, olce, Koine Yuusuke <koinec@yahoo.co.jp>
Fixes:  https://cgit.freebsd.org/src/commit/?id=87ba088fa310 ("x86/local_apic.c: Add support for installing a thermal interrupt handler")
Differential Revision:  https://reviews.freebsd.org/D58332

hwpstate_intel: Fix i386 build

Reviewed by:    olce
Fixes:  https://cgit.freebsd.org/src/commit/?id=7b26353a59d6
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58208

hwpstate_intel: Minimize ifdef for i386 build

Reported by:    jrtc27
Fixes:  https://cgit.freebsd.org/src/commit/?id=bdc0f7678257
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation

aq(4): expand and correct offloads, fix VLAN/multicast filtering

Advertise the offloads the hardware already performs, correct the TX
descriptor's L3 family selection, and correct the VLAN and multicast
receive-filter paths.

Offloads: advertise IFCAP_HWCSUM_IPV6 (adding CSUM_IP6_TCP/UDP/TSO to
isc_tx_csum_flags) and IFCAP_VLAN_HWTSO, and enable the RX outer
(S-VLAN) tag parse mode in aq_hw_offload_set().

TX descriptor L3 family: aq_setup_offloads() derived tx_desc_cmd_ipv4
from CSUM_IP|CSUM_TSO, but CSUM_TSO is (CSUM_IP_TSO|CSUM_IP6_TSO) and
tcp_output() sets both bits without regard to address family, so an
IPv6 TSO frame matched on CSUM_IP_TSO and went out with the IPv4
header-checksum command set on a frame that carries no IPv4 header.
The checksum flags cannot distinguish the family; key the bit off
IPI_TX_IPV4 instead, which iflib derives from the parsed ethertype,
as the IPI_TX_INTR test below it already does.  Plain IPv6 checksum
offload was unaffected, as CSUM_IP6_TCP alone never matched the mask.

RX VLAN tag stripping: ring init hardwired hardware tag stripping off
while the RX path still set M_VLANTAG and the writeback tag for every
tagged frame, so a tagged frame arrived with the tag in line while the
mbuf claimed it stripped and ether_demux() parsed four bytes short of
the payload.  Program per-ring stripping from IFCAP_VLAN_HWTAGGING and
set M_VLANTAG only under the same capability, so the two states stay
coherent.

VLAN filter and promiscuous edge cases: filter only when 1..16 VLANs are
registered -- with none (or more than the 16 the table holds) fall back
to VLAN-promiscuous and pass all tags, rather than dropping every tagged
frame against an empty filter table; and keep VLAN-promiscuous set
whenever the interface is IFF_PROMISC, so adding or removing a VLAN under
promisc does not clear it and start dropping tagged frames.

Multicast reconcile: ifdi_multi_set is declarative, but aq_if_multi_set()
only added -- shrinking the list left accept-all-multicast latched or
stale exact slots enabled, defeating hardware multicast filtering until
a reinit.  Clear the exact slots before reprogramming the current list,
and always drive accept-all-multicast from the current state so a shrink
clears it.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58145

aq(4): drop errored RX frames instead of resetting the interface

aq_isc_rxd_pkt_get() returned EBADMSG when a receive descriptor's
MAC/receive-error bit (rx_stat bit 0) was set.  iflib treats any error
from isc_rxd_pkt_get() as a fatal ring fault and answers with
IFC_DO_RESET -- a full interface reinitialization.  A per-frame receive
error is not a ring fault: on a marginal link or cable the Atlantic
delivers errored frames continuously, so each one triggered another
reset and the interface reset-stormed itself into carrying no traffic
instead of merely dropping the bad frames.

The Atlantic delivers errored frames to the host by design (Linux drops
them in software via buff->is_error), and iflib offers no per-frame
error return that isn't a reset.  Follow the vmxnet3 model: on a receive
error zero the fragment lengths and return success.  iflib then discards
the packet (assemble_segments() excludes zero-length fragments) while
still recycling the descriptors through the refill path -- no reset.
Also drop frames flagged with an RX-DMA fault (rdm_err), not just the
MAC-error bit; and keep iri_len non-zero on that drop path, since iflib
asserts iri_len != 0.

The genuinely structural errors -- more segments than isc_rx_nsegments,
or a pkt_len inconsistent with the descriptor count -- still return
EBADMSG, since those indicate a confused ring where a reset is the right
recovery.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58136

aq(4): honor the kernel RSS policy and add a TX traffic-class helper

Align RX steering with the kernel RSS framework and factor out the
active-traffic-class count.

RSS key and indirection table: on an options RSS kernel the stack owns a
canonical hash key and a hash-to-bucket indirection table binding each
bucket to a CPU.  aq programmed a random arc4rand() key and a plain
i % rss_qs table, so the hash it stamped in iri_flowid and the queue it
steered a flow to did not match the CPU the stack chose -- defeating RSS
affinity.  Under #ifdef RSS take the key from rss_getkey() and each entry
from rss_get_indirection_to_bucket(), as e1000/ixgbe/ixl do; the non-RSS
build keeps the random key and round-robin table.

RSS hash-type policy: drop the private hw.aq.enable_rss_udp knob (RDTUN,
default on) and add aq_rss_hashconfig(), which under options RSS returns
rss_gethashconfig() and otherwise the same UDP-off default.  UDP 4-tuple
hashing scatters a fragmented datagram's pieces across queues because
only the first fragment carries the L4 ports, so it is now off by default
and re-enabled the standard way, via net.inet.rss.udp_4tuple, matching
ix/ixl/mlx5.  On Atlantic 1 the UDP-off action stays the existing L3L4
flow-filter workaround; only its policy source changes.

TX traffic-class helper: factor the active-TC count (one per active
8-ring group, capped at HW_ATL_B0_TCS_MAX) out of aq_hw_qos_set() into
aq_hw_active_tcs(), so there is a single definition of the policy; the
Atlantic 2 RSS redirection table reuses it.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58137

aq(4): harden the attach, detach, and reset error paths

Correct several attach/detach/reset paths that either swallowed failures
or acted on undefined state.

MSI-X attach-failure double-free: aq_if_msix_intr_assign() freed the
per-RX-ring interrupts in its failure path and then returned an error, so
iflib's IFDI_DETACH freed the same irq structures again --
bus_teardown_intr() on a dangling tag and bus_release_resource() on an
already-released IRQ, panicking a box that should have simply failed to
attach.  Let iflib own the teardown; drop the failure-path loop and the
now-dead index bookkeeping.

Detach loop bound: aq_if_detach() freed the per-ring interrupts looping
to isc_nrxqsets while indexing rx_rings[], which is sized by
rx_rings_count; index by rx_rings_count to match every other RX-ring
loop.

AQ_HW_WAIT_FOR final poll: the macro derived its result from the loop
counter rather than the condition, so a condition that became true on the
last iteration reported ETIMEDOUT.  Worst for the acquire-on-read
firmware RAM semaphore, which was acquired in hardware but reported as a
timeout.  Return based on the last evaluation of the condition.

RBL MAC reset SPI cleanup: mac_soft_reset_rbl() fired the global reset
without first tearing down the SPI/flash interface, so a flash burst in
flight left the SPI bus wedged, the RBL could not re-read flash, and the
reset returned EBUSY -- fatal at attach ("MAC reset failed: 16").  Set
bit 4 of the SPI control register (0x53c) before the global reset, as the
sibling FLB path and the Linux driver do.

Reset failure propagation: aq_hw_reset() discarded fw_ops->reset()'s
return, so a failed attach-time fw2x capability read left fw_caps == 0
permanently and stats silently froze.  Propagate the error so the reset
fails and is retried.

aq_hw_init failure propagation: aq_hw_init() discarded
aq_hw_init_tx_path()/aq_hw_init_rx_path() returns and reported success,
bringing the interface up half-initialized; capture both and goto
err_exit (mainly the Atlantic 2 RX action-resolver path, which returns
EBUSY on ART semaphore timeout).

Link-state outputs: aq_hw_get_link_state() left *link_speed and *fc_neg
unwritten on early-return paths, and the caller acts on them
uninitialized, so a transient firmware get_mode() failure could fabricate
a phantom link-up at a garbage speed and program a garbage RX-pause bit.
Initialize both to safe link-down values before calling get_mode().

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58138

aq(4): harden the interrupt and MAC-statistics paths

Firmware-statistics accounting and interrupt-routing fixes.

Stats delta underflow: guard the MAC statistics delta accumulation
against counter wrap or a firmware counter reset, so a snapshot smaller
than the previous one does not underflow into a huge spurious delta.

Skip stats on a failed read: aq_update_hw_stats() ignored
aq_hw_mpi_read_stats()'s return and committed the on-stack mbox into
last_stats unconditionally.  On a failed read that snapshot is garbage or
zero and poisons the delta baseline (a zeroed snapshot wipes last_stats,
so the next good read double-counts).  Check the return and skip the
accumulation and the last_stats commit on failure.

Mailbox/stats separation: struct aq_hw_stats served both as the raw fw1x
MCP mailbox layout and as the driver's canonical stats snapshot, so any
field added to it would silently shift the fw1x mailbox read.  Give the
fw1x mailbox its own raw layout in struct aq_hw_fw_mbox and let
aq_hw_stats become purely driver-owned; with the coupling gone, add
first-class aggregate octet fields (brc/btc) that Atlantic 2 B0 firmware
can populate directly.  No A1 behavior change.  The raw block is a named
struct (aq_fw1x_mbox_stats) with a _Static_assert tying its size to
aq_hw_stats' matching prefix, so the fw1x memcpy cannot silently misalign
if either field list drifts.  Also drop the unused FW1X_MPI_STATE_ADR /
FW1X_MPI_CONTROL_ADR macros and the redundant fw1x_get_stats() dpc
assignment that the caller immediately overwrites.

Per-speed interrupt moderation: aq_hw_interrupt_moderation_set()
hardcoded speed_index = 0, so every link speed got the 10G timer pair and
the other rows were dead.  Record the negotiated rate and index the
tables by ffs(speed) - 1, reordering the rows to match the
enum aq_fw_link_speed bit positions so the index cannot drift from the
enum.  Rename the two per-speed timer tables (AQ_HW_NIC_timers_table_
{rx,tx}_ -> aq_itr_timers_{rx,tx}), function-local static arrays whose
SCREAMING_CASE vendor names read like macros.

Hardware error interrupts: route both hardware error causes (interrupt
map register 0) to the admin vector so they are actually delivered.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58139

aq(4): remove dead code and tidy macros, diagnostics, and naming

Non-functional cleanup, with two diagnostic corrections.

Dead code: delete leftover commented-out AQ_DBG_ENTER/EXIT/PRINT calls
(aq_hw.c, aq_fw2x.c, aq_irq.c, aq_main.c), a commented-out aq_nic_cfg
local, the stale old-signature parameter blocks between the ring-init
declarations and their bodies (aq_ring.c), a trailing note on a live
statement, and the unused DumpHex() vendor debug helper (no callers; its
body only compiled under AQ_CFG_DEBUG_LVL > 3).

Register-write macros: parenthesize AQ_WRITE_REG_BIT's msk/shift/value
arguments so a compound argument cannot mis-bind, give AQ_HW_FLUSH() an
explicit hw parameter instead of capturing it from caller scope, and drop
the duplicate lowercase aq_hw_write_reg[_bit] aliases (converting the 43
call sites to the uppercase spelling) so there is a single form.

Diagnostics: the aq_log* family expanded through the base log macro, which
ignored its level and printed unconditionally, while the error traces
gated on a debug level that defaulted below LOG_ERR and so were suppressed
-- backwards.  Gate the base log macro the way the trace one does and
default the level to lvl_error, so the once-per-event firmware reset /
capability errors are visible by default while the verbose info/dump
output stays opt-in.

Naming: rename identifiers carried verbatim from the vendor import that do
not match style -- names mixing an ALL-CAPS macro-style prefix with a
lowercase tail, and a trailing underscore the vendor used as a
"file-local" marker in place of static.

- dbg_level_ / dbg_categories_ -> aq_dbg_level / aq_dbg_categories: these
  are real globals (the log/trace macros reference them from every
  translation unit), so the trailing underscore was never a stand-in for
  static; give them the aq_ namespace so the driver stops exporting
  generically-named global symbols.
- log_base_ / trace_base_ -> aq_log_base / aq_trace_base: the internal
  macros behind the aq_log*/trace* families.
- bootExitCode / flbStatus -> boot_exit_code / flb_status (aq_fw.c);
  flb_status now matches the identically-purposed variable already spelled
  that way in the sibling FLB-reset path.

Cosmetic: terminate the ring/HW-init, MSI-X admin-handler, and
media-change error messages with a newline so they are not garbled into
adjacent dmesg output, and label the per-queue rx_bytes sysctl "RX Octets"
(it was copy-pasted "TX Octets").

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58140

aq(4): add Atlantic 2 (AQC113) device support

Add support for the Marvell Atlantic 2 (AQC113/114/115/116) controllers,
a new chip generation that is not register-compatible with the Atlantic 1
parts aq(4) supports today.  Adapted from the OpenBSD/NetBSD if_aq driver.

Register and device definitions (aq2_hw.h): the firmware handshake
(MIF_BOOT / MCP_HOST_REQ_INT / MIF_HOST_FINISHED), the 0x12000/0x13000
firmware interface windows, and the action-resolver table (ART) that
replaces Atlantic 1's discrete RX filters, plus the Atlantic 2 PCI device
ids and the aq_is_atlantic2() helper.  Reserve a chip-feature bit
(AQ_HW_CHIP_ATLANTIC2) and add the aq_hw fields the firmware fills at boot
(ART base index, statistics interface version A0/B0).  The per-VLAN-filter
resolver-tag field comes from the Linux driver; the BSD sources never
write it.

Firmware operations (aq_fwa2.c): Atlantic 2 talks to the management CPU
through the 0x12000/0x13000 register windows plus the boot handshake,
rather than Atlantic 1's mailbox in shared RAM.  Implement that as a third
aq_firmware_ops vtable (reset, set_mode, get_mode, get_mac_addr,
get_stats); aq_fwa2_reboot() boots the firmware, selects the A2 ops, and
reads the version and ART base index, failing fast on the
crash-init / boot-failed bits.  fwa2_set_mode advertises full duplex only
(the media model exposes no half-duplex types) and writes and acks the
link options before raising ACTIVE mode, so a forced media change does not
begin negotiation with a stale rate mask.  enum aq_fw_link_speed gains
aq_fw_10M, which Atlantic 2 supports and Atlantic 1 does not.

Probe and attach: list the device ids with their media types and link
speeds (all copper; AQC113* up to 10G, AQC116C to 1G), populate
hw->device_id, and tag the generation with AQ_HW_CHIP_ATLANTIC2 so
IS_CHIP_FEATURE() recognises it uniformly.  Branch firmware bring-up and
reset on the generation: aq_hw_init_ucp() and aq_hw_reset() reboot the MCP
instead of the Atlantic 1 RBL/FLB reset -- without a real datapath reset
every stop/init cycle reprograms the rings on a live, desynced RX DMA
engine and the receive path stays dead.  aq_hw_init() programs the
Atlantic 2 launch-time clock ratio in place of the Atlantic 1
MRRS / TX-DMA request-limit clamp.  Add an AQ_LINK_10M capability bit
(Atlantic 2 links at 10M, Atlantic 1 cannot), offer 10baseT media, and map
IFM_10_T to aq_fw_10M.

With every supported media type now present, replace the per-speed switch
statements in aq_media.c with a single {link bit, fw rate, IFM_* subtype,
Mbit/s} table -- one source of truth for the supported link speeds.

With this an Atlantic 2 card probes, brings up its firmware, reads its
MAC, and negotiates link; the RX action-resolver datapath comes next.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58141

aq(4): program the Atlantic 2 multiqueue datapath

Wire up the Atlantic 2 receive datapath: the action-resolver table (ART),
multiqueue RSS, QoS, and interrupt moderation.

RX action-resolver table: Atlantic 2 replaces Atlantic 1's discrete RX
filter registers with an ART -- hardware computes a per-packet
classification tag, then walks {tag, mask, action} rows to drop, assign a
queue, or assign a TC.  aq_hw_art_filter_set() installs one row under the
ART semaphore; aq_hw_init_rx_path() enables the resolver, tags L2
unicast/broadcast, installs the unicast/all-multicast and VLAN drop rows,
and assigns every 802.1p priority to TC 0 (mirroring the Atlantic 1
user-priority map, since our RX side is a single 8-ring group in TC 0).
Tag every enabled VLAN filter in the per-filter resolver-tag field -- a
register the BSD ports never write -- because the VLAN drop row matches
resolver tag 0, so without it all tagged receive was dead under VLAN
filtering.  Promiscuous mode disables the drop rows rather than toggling
the Atlantic 1 promiscuous bits; all ART callers surface a semaphore
timeout consistently.  The Atlantic 1 RX_TCP_RSS_HASH and TPO2
programming is gated to Atlantic 1.

Multiqueue RSS and QoS: fill Atlantic 2's own per-TC redirection table
(AQ2_RPF_RSS_REDIR), skipping the Atlantic 1 table and its write-enable
handshake.  Program Atlantic 2's smaller packet-buffer sizes, its wider
data-TC credit/weight fields, and its ring-to-TC map, using
aq_hw_active_tcs() for the TC loops.

RSS hash types: the Atlantic 2 resolver has per-protocol hash-type enables
in REDIR2, so build the mask from aq_rss_hashconfig() instead of
hardcoding every protocol -- UDP 4-tuple hashing now follows the kernel
policy (off by default) with no L3L4 flow-filter workaround, and
aq_hw_udp_rss_enable() is skipped on Atlantic 2.  The kernel-to-hardware
hash-type mapping is a small static lookup table rather than a nine-branch
chain, since the two bit spaces do not share a simple shift.

Tx interrupt moderation: Atlantic 2's per-ring Tx moderation control
register lives at a different address, but its field layout matches the
value the driver already builds, so write that value straight to it; Rx
moderation is shared.

HW-validated on AQC107 <-> AQC113C: TCP RSS spreads across 7/8 RX queues
under 16 parallel flows, rx_err=0.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58142

aq(4): correct Atlantic 2 register access

Four Atlantic 2 register-access corrections found in bring-up.

B0 aggregate octet counters: the B0 firmware statistics interface reports
only aggregate rx/tx good octets, not the per-cast breakdown A0 and
Atlantic 1 provide, so every octet sysctl read a permanent zero while
frame counters advanced.  Populate the aggregate octet fields from the B0
buffer; aq_update_hw_stats() accumulates them directly when the per-cast
octets are absent.

Drop the duplicate attach-time MCP reboot: aq_hw_mpi_create() already
reboots the A2 firmware to read its version and caps, then aq_hw_reset()
immediately rebooted it again -- a full MCP restart plus several
transaction-id-bracketed window reads, adding attach latency and a
duplicate banner.  Give aq_hw_reset() a reboot flag and pass reboot=false
for A2 at attach; the load-bearing down/stop reboot (which resyncs A2 RX
DMA across ifconfig down/up) keeps reboot=true.

Skip Atlantic 1 register accesses on Atlantic 2: gate out the 0x7040
Atlantic 1 TPO write (which A2 lacks; already a no-op via the unset TPO2
feature, but Linux hw_atl2 omits it), and guard the
aq_hw_mpi_read_stats() direct reads of reg_rx_dma_stat_counter7 (dpc) and
the LRO counter (cprc) with !ATLANTIC2 -- those are Atlantic 1 codegen
offsets that on Atlantic 2 land on unrelated registers and can report
bogus input-drop / LRO counts.

HW-validated on AQC107 <-> AQC113C: A1 stats unchanged, A2 IQDROPS stays
0, attach consumes one MCP reboot instead of two, bidirectional iperf3
clean.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58143

aq(4): observability controls and sysctl/header hygiene

Fold the driver's observability and infrastructure work.

Make aq_device.h self-contained: it declares struct aq_dev in terms of
iflib, bitstring, socket, and ethernet types but included none of the
headers that define them, compiling only because every includer happened
to pull those first.  Include what it uses.  No functional change.

Make the debug controls per-instance.  The debug and debug_categories
sysctls were registered per device but pointed at file-scope globals, so
writing dev.aq.1.debug also changed dev.aq.0.debug and a card could not
be traced in isolation.  Move the level and category mask into struct
aq_dev, reach them through the aq_dev back-pointer in struct aq_hw (wired
up in attach_pre before the first firmware trace and guarded against a
NULL deref), emit through device_printf() so each line carries its unit,
and seed initial values from per-unit device hints so attach can be
traced.

Expose the PHY die temperature as dev.aq.N.temperature through a new
firmware get_temp op: Atlantic 1 v2 reads it through the mailbox MPI
control/state toggle, Atlantic 2 from the phy_health_monitor block in the
OUT window (located at 0x13620 and confirmed by its ready bit).  Atlantic
1 v1 has no sensor and exposes no node.  Because this is the first
firmware accessor iflib does not serialise, add a per-instance mutex in
struct aq_hw and take it across the v2 read-modify-write in set_mode(),
get_stats(), get_mode(), and get_temp(); the v1 and Atlantic 2 paths do
not need it and say so.

Trace the Atlantic 2 firmware path, which previously emitted nothing at
any debug level (aq2_fw.c did not even include aq_dbg.h): the boot
handshake, reset policy, MAC address, and link mode set/read, using the
existing dbg_init and dbg_fw categories, with the per-poll mode read at
detail level.

Scope the driver sysctls to a context freed at detach.  They were
registered on the device newbus context, which newbus tears down only
after DEVICE_DETACH returns, yet iflib frees the rings and softc inside
DEVICE_DETACH -- a sysctl read racing detach could touch freed memory.
Give the driver its own sysctl_ctx_list and free it at the start of
aq_if_detach, draining in-flight readers first.

Signed-off-by: Nick Price <nick@spun.io>

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58434

aq(4): PHY thermal-shutdown handling and correctness fixes

Fold the thermal-protection work and the correctness fixes that landed
alongside it.

Report and auto-recover from PHY thermal shutdown.  The Atlantic PHYs can
autonomously shut down on over-temperature, latching global fault 0x8007
and dropping the link; Atlantic 2 ships this armed, Atlantic 1 disabled.
Arm it on Atlantic 1 at interface init (1E.C478.A via the MAC's MDIO
controller), and recover from a trip automatically: the admin-status poll
detects the fault, logs the shutdown limit and measured temperature, and
holds the link down until the PHY cools, then restores it -- Atlantic 1
needs a PHY reset (1E.2681.0) with the MAC firmware running plus a full
re-init, Atlantic 2 recovers on the re-init alone.  New firmware ops
get_phy_fault, phy_reset, thermal_arm, and get_thermal_limit back the
state machine in aq_if_update_admin_status().

Make that Atlantic 1 thermal MDIO path address-correct and fail-safe.
The direct-MDIO helpers hardcoded the Clause-45 port address to 0, but it
is strap-selectable: on a board whose PHY answers elsewhere every thermal
op targeted nothing, so arming silently no-oped and the post-trip reset
never cleared the latch.  Discover the address by scanning ports 0..31
for a PMA/PMD identifier and form it as (phy_id << 5) | mmd, marking it
valid only when a PHY actually answers.  aq_fw2x_phy_read also returned 0
on a semaphore timeout, indistinguishable from a real 1E.C478 == 0, so
thermal_arm could zero live provisioning bits; give the read an error
return and gate thermal_arm and get_thermal_limit on it.

Bound the multicast filter slot index.  aq_mc_filter_apply() programmed
slot count + 1 and bailed only at count == AQ_HW_MAC_MAX (33), one
address too late, so a 33rd entry raced in between the if_llmaddr_count()
snapshot and the if_foreach_llmaddr() walk drove an out-of-bounds MMIO
write to slot 33.  Fire the guard at AQ_HW_MAC_MAX - 1, and also reject
index >= AQ_HW_MAC_MAX in aq_hw_mac_addr_set() where the slot becomes an
RPF register offset.

Correctness and safety fixes: initialize the sysctl context in
attach_pre so the iflib fail-path detach cannot sysctl_ctx_free() an
uninitialized list (a page fault when MSI/MSI-X is denied); range-check
the Atlantic 2 action-resolver table index, taken verbatim from a
firmware-supplied base, before writing the ART registers; and accumulate
statistics deltas as unsigned, since AQ_SDELTA discarded a forward delta
of 2^31 or more at 10G across a stretched admin poll.

Signed-off-by: Nick Price <nick@spun.io>

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58435

aq(4): clean up diagnostics and remove dead code

Non-functional cleanup, no change in behavior.

device_printf() already prefixes each line with the device name, so the
inline "atlantic:" token in the status and error messages produced a
doubled prefix and diverged from the trace macros; remove it so all
output carries one uniform "aqN:" prefix.  Compile the RX/TX descriptor
tracers only when AQ_CFG_DEBUG_LVL > 2 and make them no-op macros
otherwise, so the default build no longer pays a cross-TU call plus
argument evaluation per descriptor.

Drop enum aq_dev_state, struct aq_rx_filters, and struct aq_vlan_tag,
which have no remaining references now that VLAN state lives in a
bitstr_t.  Replace the four identical aq_sysctl_print_{tx,rx}_{head,tail}
handlers, each carrying a dead write path on a read-only oid, with one
aq_sysctl_print_ring_ptr that selects the accessor from arg2.  Reduce the
thermal and PHY-recovery comments to single terse lines that keep the
load-bearing register numbers and the A1-vs-A2 recovery difference.

Signed-off-by: Nick Price <nick@spun.io>

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58436

aq(4): mailbox, flow-control and firmware error-handling fixes

Fold the whole-driver-review correctness and hardening fixes for the
firmware and hardware layers.

Advance the firmware-mailbox address per word in aq_hw_fw_downld_dwords():
on B1 silicon each loop iteration waits for the mailbox address register
to differ from the expected address, but it was set once and never moved,
so after the first word every wait returned immediately and read stale
data.  Advance it four bytes per word.  B0 is unaffected (it polls the
busy bit).  The same function also left err set to ETIMEDOUT after
successfully force-recovering the RAM CPU semaphore; the transfer loop is
guarded by "--cnt && !err", so it ran zero iterations and returned a
timeout with an untouched buffer, making the recovery path dead code.

aq_hw_get_mac_permanent() ignored the get_mac_addr() error and then
examined a buffer the firmware op never wrote on failure.  A fresh softc
is zero, so the "invalid address" test fired, a random locally
administered MAC was substituted, and err was overwritten with 0 -- a
transient mailbox failure produced a card that attached with a different
MAC every boot.  Fail instead; the random-address fallback still covers a
genuinely blank or multicast burned-in address.  aq_fw1x_reset()
discarded the same download's return value and then read transaction_id
out of an uninitialized stack struct, so propagate that error too.

Encode RX-only flow control as PAUSE|ASYM_PAUSE rather than PAUSE alone:
firmware 2.x/3.x has no independent RX-only bit, so the old encoding
advertised symmetric pause when RX-only was requested.  The MPI_INIT path
also never cleared the pause bits before OR-ing in the requested ones, so
flow control could be enabled and never disabled; clear them first, as
the Atlantic 2 and Linux implementations do.

Reject single-vector MSI in aq_if_attach_post() the same way legacy INTx
is rejected: ift_legacy_intr is NULL, so no driver filter would
acknowledge the not-clear-on-read, auto-masked device interrupt status;
every supported Atlantic device provides MSI-X.

Propagate firmware and MDIO errors instead of discarding them.  The fw2x
MDIO primitive returned a data word with no way to report a controller
timeout; give aq_fw2x_mdio_op() a status return and a data out-parameter,
propagate it through phy_write/read/reset/thermal_arm, and stop advancing
the thermal recovery state machine when a PHY reset fails.  Use that
error to end the PHY address scan early: aq_fw2x_init_phy_id() probed all
32 MDIO ports even when the controller itself was timing out, spending up
to ten seconds under fw_mtx and the iflib context lock.  aq_fw2x_reset()
also drove the shared MIF mailbox without fw_mtx, unlike every other fw2x
mailbox user, so it could interleave with the temperature sysctl and load
the capability mask from the wrong window.

aq_hw_mpi_set() can return ETIMEDOUT when the Atlantic 2 shared firmware
buffer is not acknowledged; aq_hw_init() now aborts through its error
path rather than enabling rings with an unaccepted link state, and
aq_if_init() logs the later link-speed error.

Retry a failed initialization instead of leaving the link down.  ifdi_init
has no return value, so iflib marks the interface running once aq_if_init()
returns; a propagated firmware-ack failure would otherwise leave it running
with no initialized hardware and no recovery.  Record the failure and retry
from the admin task via iflib_request_reset(), paced by the once-per-second
timer, giving up after a bounded number of attempts.  Ring and queue start
failures are deliberately left to the existing diagnostic, since they leave
the remaining queues usable.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58437

aq(4): interface lifecycle and link-state fixes

aq_if_init() programmed the address captured at attach, so an address set
with "ifconfig ether" or by lagg(4) enslavement was never written to
unicast filter slot 0: the interface transmitted with the new address but
the MAC still filtered on the old one, so it received nothing.  Copy the
current if_getlladdr() the way the other iflib drivers do.

The link state could latch UP forever.  aq_if_stop() cleared linkup
before calling aq_if_update_admin_status(), which suppressed the
LINK_STATE_DOWN transition the "link was UP" branch would have made.
Announce the down transition directly from aq_if_stop() instead, and do
not poll the admin status there at all: the MAC has just been reset, so a
stale link reading would re-announce the link as up.

The admin task itself had to stop reporting a link on a stopped
interface.  iflib runs it while either IFF_DRV_RUNNING or IFF_DRV_OACTIVE
is set, and iflib_stop() sets OACTIVE, so the task kept polling after the
stop and re-announced LINK_STATE_UP behind the driver's back.  Treat a
non-running interface as having no link.  A lagg(4) parent otherwise
keeps hashing flows onto a port whose carrier is gone, because
LAGG_PORTACTIVE tests if_link_state together with IFF_UP.  Stop the rest
of the task there as well: the PHY thermal poll and the initialization
retry both end in iflib_request_reset(), and _task_fn_admin() acts on
that with no test of its own, so either could re-initialize an interface
the operator had just taken down.

aq_if_update_admin_status() also only reacted to transitions in and out
of zero speed, so an autoneg downshift that kept the link up left
if_baudrate, ifmedia, RX pause and interrupt moderation programmed for
the old speed.  Track the announced speed and re-run that work when it
changes.

aq_if_suspend() resets the MAC and stops the rings, but
iflib_device_suspend() only calls IFDI_SUSPEND and never stops the
interface, leaving IFF_DRV_RUNNING set over a suspended device.  Clear it.

Signed-off-by: Nick Price <nick@spun.io>

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58473

re(4): quiesce RTL8168G+ and reset before freeing buffers in re_stop()

The STOPREQ command written by re_stop() is not defined for
RTL8168G and later; issuing it can wedge the MAC.

Replace it on those parts with the vendor-documented sequence:

* settle delay
* bounded poll for Tx queue empty
* clear TE/RE
* then bounded poll of the MCU command register (0xD3) FIFO-empty bits.

Also reset the controller before the Rx/Tx buffer free: a controller that
has not quiesced keeps DMAing stale, still-owned descriptors pointing at
freed mbufs (use-after-free under INVARIANTS, cross-NIC mbuf corruption
reported in the PR).

Adds the RL_MCU_* register definitions.

All waits are bounded; error paths only.

* iperf3 --bidir at line rate against RTL8168H (XID 0x541);
  previously wedged the controller until power cycle, with the
  quiesce the reset path recovers.

* Deployed in production on an RTL8168H fleet since 2026-07-01.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58276
PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724

re(4): re-arm the Tx doorbell when re_txeof() leaves a non-empty ring

On PCIe parts a TxPoll request can be lost when packets are queued
in quick succession, leaving owned descriptors with no transfer in
progress until the watchdog fires. re_txeof() runs from the interrupt
handlers, re_tick() and re_watchdog(), so re-writing TXSTART whenever
the ring is still non-empty turns a potential 5-second stall into at
most one tick.

One register write on a path that already took an interrupt;
fast path untouched.

* Sustained bidirectional load on RTL8168H; no Tx stalls, no throughput
  regression at 941 Mbps line rate.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58277
PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724

re(4): recover Tx completions whose MSI was swallowed in re_intr_msi()

A Tx completion that raises a status bit between the ISR ack at the top
of re_intr_msi() and the IMR re-enable at the bottom is never re-signalled:
these controllers do not re-assert MSI for an already-set status bit
(this is why hw.re.msi_disable is a known workaround in the PR).

Re-read ISR before re-enabling; if a Tx bit is pending, ack just that bit,
reap the ring and restart the queue. Rx bits are deliberately left set so
they re-arm the interrupt normally and Rx moderation state is untouched.

Also flush the posted IMR write. Mirrors what the INTx path already
achieves via the loop in re_intr().

* MSI interrupt mode on RTL8168H under load; "missed Tx interrupts"
  watchdog recoveries no longer occur.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58278
PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724

re(4): harden re_watchdog() recovery and log controller state

Distinguish the two failure classes from the PR in a single log line
(ring indices, ISR/IMR, TXCFG, interrupt mode): lost interrupt vs
genuine DMA stall.

Bail out instead of re-initializing when the controller reads back
all-ones (fallen off the bus; reinit cannot help).

Re-assert the driver's existing ASPM-disabled policy before reinit,
since firmware/power transitions re-arming L0s/L1 is a documented
stall trigger.

Diagnostics-only on the recovered path; no fast-path change.

* Field diagnostics running on an RTL8168H production fleet; the log
  format distinguishes lost-doorbell / DMA-stall / dead-controller
  without a debug build.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58279
PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724

re(4): add hw.re.aspm_disable loader tunable

re(4) has unconditionally disabled ASPM L0s/L1 and CLKREQ at attach
for years; on laptops this costs 200mW+ (requested by adrian@ in the PR).

Make it a tunable following the existing hw.re.* pattern:

* default 1 keeps today's behavior;
* 0 preserves the firmware-configured ASPM state at attach and
  skips the watchdog re-assert from the previous revision.

Documented in re.4.

* Verified on RTL8168H (XID 0x541): with hw.re.aspm_disable=0,
  attach no longer logs "ASPM disabled" and pciconf -lcb shows
  the firmware Link Control state preserved -- including Clock PM,
  which the unconditional code previously cleared.

* Default (1) is behaviorally identical to the current driver.

* Note the tunable also stops the driver clearing CLKREQ, a small power
  win even where firmware leaves L0s/L1 off.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58280
PR: kern/https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166724

Provide pc_small_core for i386 too to fix an i386 build break from x86
code referring to it.  It won't be set.

Reviewed by:    aokblast, kib
Fixes:  https://cgit.freebsd.org/src/commit/?id=7b26353a59d6 ("hwpstate_intel: Disable package control on hybrid CPU")
Differential Revision:  https://reviews.freebsd.org/D58335

bus_{read,write}_8 are macro wrappers around the corresponding bus_space
functions in sys/bus.h, so implementing bus_{read,write}_8 won't work.
Implement the underlying bus_space function instead.

Reviewed by:    jrtc27, rlibby
Fixes:          https://cgit.freebsd.org/src/commit/?id=9313f6b01485
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58301

The accumulated count of a process-mode counting PMC is kept in a
64-bit software counter and seeded into the hardware counter at every
context switch in.  Hardware counters are narrower than that - each
PMC class discovers and records its own counter width, e.g. 48 bits
on current x86 (queried from CPUID on Intel, architectural on AMD) -
so once the accumulated count approaches the end of the hardware
counter range, the counter wraps during a time slice and the value
read back at switch out is smaller than the value seeded.  The
increment was computed assuming a full 64-bit counter: on INVARIANTS
kernels a long enough counting run panics with "negative increment"
the moment the accumulated count first crosses the hardware counter
range, and on other kernels the totals silently lose a full counter
range per wrap.

Compute the increment modulo the per-class hardware counter width
instead, in both places that accumulate switch-out deltas.

Reviewed by:            adrian
MFC after:              2 weeks
Assisted-by:            Claude Code (Fable 5)
Differential Revision:  https://reviews.freebsd.org/D58340

A process-mode PMC's runcount tracks how many CPUs currently have it
loaded in hardware.  It is decremented only by the context-switch-out
and process-exit reclaim paths, both of which the scheduler invokes
only for processes flagged P_HWPMC.  Detaching a target that still has
the PMC live in hardware dropped the target and cleared P_HWPMC without
taking the PMC off the hardware or dropping the runcount reference, so
the reference leaked.  A subsequent release then spun in
pmc_wait_for_pmc_idle() forever waiting for the runcount to reach zero:
on an INVARIANTS kernel this panics ("waiting too long for pmc to be
free"), otherwise it is an unkillable loop holding the hwpmc lock.  Any
process able to allocate a PMC can trigger this by attaching a counting
PMC to itself and detaching it before releasing.

Take the PMC off the hardware and drop the runcount reference as part
of detaching, before P_HWPMC is cleared: reclaim it from the detaching
thread's own CPU directly, and, when the detach removes the PMC's last
target, wait for any references held by the target's other threads to
drain while P_HWPMC is still set (they can no longer reload it).

Reviewed by:            adrian
MFC after:              2 weeks
Assisted-by:            Claude Code (Fable 5)
Differential Revision:  https://reviews.freebsd.org/D58342

Some UVC devices (e.g. Logitech C920) expose more than 8 Processing
Unit descriptors, causing "too many PU descriptors found!" errors.
Increase both limits from 8 to 32 to accommodate such devices.

Import the device quirk system from OpenBSD to handle UVC devices
that need special handling. This includes:

- UVIDEO_FLAG_ISIGHT_STREAM_HEADER: non-standard streaming header
- UVIDEO_FLAG_REATTACH: needs reattach after firmware upload
- UVIDEO_FLAG_VENDOR_CLASS: incorrectly reports as vendor class
- UVIDEO_FLAG_NOATTACH: device not supported
- UVIDEO_FLAG_FORMAT_INDEX_IN_BMHINT: format index in bmHint

Add quirks table with known devices and lookup function.
Add iSight stream header decoder for Apple iSight cameras.

Obtained from:  OpenBSD

Sponsored by: The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D56005

A bunch of drivers weren't properly converted.  I mistakenly
put a call to ieee80211_output_seqno_assign() wherever the
crypto header was added, which isn't exactly correct.
There are plenty of drivers which don't share enough of their
raw and normal transmit path code for that to hold true.

So after some manual review, it looks like I've captured the
places (outside of iwn(4) which I committed earlier) where
I missed ieee80211_output_seqno_assign() calls.

* For bwi(4) and bwn(4) I refactored it out into a place that is
  common enough and happens in the same lock hold window,
  so it's serialised.

* For the rest, it's just plain missing from the raw path.

Locally tested:

* ural(4)
* ral(4)
* bwi(4)

Differential Revision:  https://reviews.freebsd.org/D58098

firewire: add warn-only CRC validation for CSR ROM directories

Implemented crom_crc_valid() helper to validate IEEE 1394 config ROM CRC-16
checksums.

Skipped root header CRC validation since csrhdr.crc_len cover the entire
ROM body which is not fully read at header parse time. Per-directory
CRC checks below catch corruption where it needed.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58307

firewire: drain pending xfers after callout stop in detach

Removes a TODO that predates the existing drain call.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58308

firewire: force root change when root node is not cycle master capable

When a FireWire bus resets, all devices negotiate who is the new boss.
when we detect the root node can't be cycle master,
we send a PHY config packet that forces a reelection.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58309

firewire: remove dead code across the subsystem

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58310

firewire: replace magic numbers with named constants

No functional change.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58311

Sponsored by: Klara, Inc.
Sponsored by:   NetApp, Inc.
MFC after:      1 week
Fixes:          https://cgit.freebsd.org/src/commit/?id=6d0001d44490 ("nvme: add support for DIOCGIDENT")
Reviewed by:    bnovkov, imp
Differential Revision:  https://reviews.freebsd.org/D58357

The uvideo driver freed the mmap buffer (contigmalloc'd) in several
paths (VIDIOC_STREAMOFF, last close, detach) without coordinating
with the lifetime of existing user-space mmap mappings.  This could
lead to use-after-free when user-space continued to access the
mapped memory after the backing pages had been freed.

Fix this by switching from the simple d_mmap callback to d_mmap_single
with custom cdev_pager_ops, and by attaching the contig buffer to a
single shared vm_object created at REQBUFS time:

- uvideo_reqbufs() allocates a uvideo_mmap_state (independent of the
  softc) and a shared vm_object via cdev_pager_allocate() that spans
  the whole buffer; the softc holds one reference to it.
- uvideo_cdev_mmap_single() simply hands out additional references to
  that shared object; the requested offset selects which buffer is
  mapped.  The VM system tracks mapping lifetime through the object
  reference count, so no per-mapping bookkeeping is needed.
- uvideo_pg_ctor/uvideo_pg_dtor validate the mapping and free the
  contig buffer together with the state when the last reference
  (softc's own or a user mapping) is dropped.
- uvideo_pg_fault installs a fictitious page for the backing physical
  address, following the canonical device-pager pattern: update the
  passed-in page in place when it is already fictitious, otherwise
  allocate a fake page and vm_page_replace() the busy placeholder,
  so that dev_pager_dealloc() does not deadlock.
- uvideo_vs_free_frame() drops the softc's reference instead of
  contigfree()'ing directly; if mappings still exist the buffer stays
  alive until the last uvideo_pg_dtor().
- VIDIOC_STREAMOFF no longer frees the buffer (per V4L2 spec).
- Last close always releases the buffer (deferred if mappings exist).
- The mmap_state outlives the softc, so the pager dtor can safely
  free the buffer even after device detach.

Reported by:    章鱼哥 (@aipyapp) (www.aipyaipy.com)
Reported by:    Chris Jarrett-Davies of the OpenAI Codex Security Team

This fixes an issue with the Solo2 (and likely some of the Nitrokey
family) where hangs would occur with OpenSSH- it issues a CANCEL prior
to closing the device unconditionally, and without draining the read
endpoint we end up seeing the response to that CANCEL the next time
OpenSSH tries to connect.  This throws the entire command/response
sequence out of whack.

This call used to break Yubikeys in some situations, but the fix that
landed in 28d85db46b48 ("xhci: Do not drop and add bits in xhci") seems
to have addressed that- presumably we sometimes end up stopping the
command and desyncing at the controller level.  This probably implies
that we need a SYNCWRITE HID quirk, but that requires a little more work
in usbhid_sync_xfer() and this doesn't seem to cause any problems in
normal usage.

Reviewed by:    aokblast, wulf
Differential Revision:  https://reviews.freebsd.org/D58199

Replace bare EINVAL in AMD/IBS allocation and config-validation with
EXTERROR(), so a failed pmc(3) allocation names the check and value.

Register HWPMC_AMD in exterr_cat.h and the generated filenames.h.

Signed-off-by:  Andre Silva <andasilv@amd.com>
Reviewed by:    Ali Mashtizadeh <ali@mashtizadeh.com>, mhorne
Sponsored by:   AMD
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2180

Annotate validation failures in the PMC syscall handlers (allocate,
attach, read/write) with EXTERROR(), so pmc(3) callers see which
precondition failed, not a bare errno.

Register HWPMC_MOD in exterr_cat.h and the generated filenames.h.

Signed-off-by:  Andre Silva <andasilv@amd.com>
Reviewed by:    Ali Mashtizadeh <ali@mashtizadeh.com>, mhorne
Sponsored by:   AMD
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2180

vtnet: Retry feature negotiation without offloads

A device is permitted to reject an otherwise valid subset of its
offered features by refusing to accept FEATURES_OK (VirtIO v1.3,
2.2.2).  Apple's Virtualization.framework does this in practice;
it treats the offered CSUM/TSO offloads as all-or-nothing, while
vtnet's default request contains only part of that group because
of hw.vtnet.lro_disable that would drop the guest TSO bits, thus
negotiation fails and the device does not attach.

If FEATURES_OK is rejected, retry the negotiation once with every
offload-related feature stripped.  Changing the feature set after
a failed FEATURES_OK requires re-initialising from device reset
(VirtIO v1.3, 3.1.1), so the retry goes through virtio_reinit().

A NIC without offloads is preferable to no NIC at all.  Devices
that accept the initial feature set are unaffected, while those
that also reject the reduced set continue to fail attachment as
before.

Signed-off-by:  Faraz Vahedi <kfv@kfv.io>
Reviewed by:    adrian
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2322

vtnet: Implement VIRTIO_NET_F_GUEST_ANNOUNCE

When the device sets VIRTIO_NET_S_ANNOUNCE in the config status
field, for example after a VM migrates to a new host, announce
the interface's presence on the network so peers and switches
learn the new attachment point, then acknowledge the request
with the VIRTIO_NET_CTRL_ANNOUNCE_ACK control command, as per
VirtIO v1.3, 5.1.6.5.4.

The announcement raises iflladdr_event: the stack sends gratuitous
ARPs and unsolicited neighbor advertisements for the interface's
addresses, and stacked interfaces such as vlan(4) propagate the
event and announce theirs as well.  The event handlers may sleep,
so the work is deferred from the config change interrupt to a task
on taskqueue_thread; that context also allows the acknowledgement
to be skipped safely if the interface was stopped in the meantime,
in which case the device keeps the bit set and the request is
re-delivered with the next config change interrupt.

Signed-off-by:  Faraz Vahedi <kfv@kfv.io>
Reviewed by:    adrian
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2322

vtnet: Accept VIRTIO_NET_F_CTRL_RX_EXTRA

Although the driver does not issue the extra receive-mode commands
accepting the feature is harmless and some devices, notably Apple's
Virtualization.framework, offer their control-queue features as a
group and refuse FEATURES_OK unless the whole set is acknowledged.

Signed-off-by:  Faraz Vahedi <kfv@kfv.io>
Reviewed by:    adrian
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2322

These kernconfs were missed in the previous commit.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=289236
Reviewed by:    kib
Fixes:          https://cgit.freebsd.org/src/commit/?id=f38cbefef8090f3363e5685c5a3b30ffbf1d3ad0
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation

uvideo: replace contigmalloc with OBJT_PHYS vm_object for mmap buffer

Allocate the mmap buffer via phys_pager_allocate() and map it into
kernel space with vm_map_find()/vm_map_wire(), instead of a custom
cdev_pager backed by contigmalloc.  phys_pager_allocate() is required
over a bare vm_object_allocate(OBJT_PHYS) to initialise un_pager.phys.ops,
otherwise phys_pager_getpages() NULL-derefs during vm_map_wire().

Reviewed by:            markj
Reported by:            markj
Differential Revision:  https://reviews.freebsd.org/D58394

uvideo: validate frame size before mmap buffer allocation

dwMaxVideoFrameSize comes from the USB probe/commit response and is not
validated.  reqbufs() computed buf_size_total with signed int arithmetic
and no bound, so a bogus value could wrap the product to a small size
and yield a too-small buffer with a huge sc_mmap_buffer_size, causing
out-of-bounds writes from the USB transfer callbacks.

Bound the frame size against sc_max_fbuf_size and use overflow-checked
size_t arithmetic for the total and per-buffer offsets.

Reported by:    emaste

uvideo: lock the mmap queue and read path

qbuf(), dqbuf() and read() manipulated sc_mmap_q / sc_mmap_cur /
sc_frames_ready without sc_mtx, racing with the USB transfer callbacks
(producer) that run under the mutex.  This could corrupt the queue or
trigger use-after-free.

Take sc_mtx around qbuf(), use mtx_sleep() and protect the queue
operations in dqbuf(), and use mtx_sleep() with a snapshot of sc_fsize
in read().

Also reject S_FMT and S_PARM with EBUSY while streaming: both
re-negotiate the probe/commit controls with the device, which disrupts
the active USB transfers (a second client opening the device would
otherwise freeze the first one's stream).

uvideo: bounds-check frame interval reads against bLength

Frame interval data is read from device-supplied frame descriptors whose
bLength may be shorter than the number of intervals declared by
bFrameIntervalType.  The continuous branch of uvideo_enum_fivals() read
three intervals unconditionally, and the discrete branch checked the
pointer but not the four bytes that UGETDW() reads, so a short or
malformed descriptor could read past bLength and leak adjacent kernel
memory to userspace.  uvideo_vs_parse_desc_frame_max_rate() had the same
class of off-by-up-to-three-bytes read.

Compute the available bytes from bLength and validate before each read.

Reported by:    emaste

uvideo: track streaming ownership per-fd and free buffers on STREAMOFF

The driver shared a single streaming state and buffer pool across all
open file descriptors, so a second client (e.g. another browser tab)
could disrupt the first: its cleanup STREAMOFF would tear down the
active stream, and stale buffers prevented re-acquisition.

Add per-fd state via devfs cdevpriv tracking whether this fd started
streaming.  STREAMOFF and close from a non-streaming fd are no-ops.
STREAMOFF from the streaming fd stops the stream and frees the buffers
so that a new fd can re-acquire the camera.  DQBUF returns EPIPE
immediately when buffers are freed instead of waiting for a timeout.

uvideo: fix close/detach race on streaming teardown

detach() stopped streaming and called uvideo_vs_close() before
destroy_dev(), so a concurrent close() could race the teardown and call
uvideo_vs_close() a second time (double usbd_transfer_unsetup), and
mtx_destroy() could race a close still holding sc_mtx.  sc_streaming
was also read without the lock in both paths.

Reorder detach() to call destroy_dev() first so all in-flight cdev
methods drain before any teardown.  Read sc_streaming under sc_mtx in
both detach() and the last-close safety net.

uvideo: use size_t for sc_mmap_count and loop index in reqbufs

uvideo: validate frame descriptors and fix integer overflows in size computation

uvideo: fix printf type

Reported by:    vishwin

Reviewed by:  aokblast, kib, olce
Differential Revision:  https://reviews.freebsd.org/D58336

Some UAC2 devices expose a single Clock Source entity that is shared
between their playback and capture interfaces (it appears in both the
output and input clock bitmaps).  On such a device uaudio(4) programs
the sample rate for both directions when a stream starts.  If playback
runs at a 44.1 kHz-family rate while the idle capture channel is left
at its 48 kHz-family default, the capture
SET_CUR(UA20_CS_SAM_FREQ_CONTROL) is issued after the playback one and
overwrites the rate on the shared clock.  The device then runs at
~48 kHz while the playback stream carries 44.1 kHz data.  Consuming
samples faster than they arrive, the device repeatedly runs out of
data, loses sync with the playback stream, and re-locks onto it
(audible dropouts, front-panel play/idle flicker).  The 48 kHz family
is unaffected because both directions then agree on the rate.

Fix it in three parts:

- Add a shared-clock guard: before issuing SET_CUR to a clock id, if
  that clock is shared between playback and capture and the other
  direction is already streaming at a different rate, skip it.  The
  first active stream owns the clock; a later one follows it.

- When the recording channel is auto-started only as a source of jitter
  information for asynchronous playback, align its nominal rate to the
  playback rate before starting it, so it neither reprograms the shared
  clock to a conflicting rate nor produces mismatched frame sizes.

- Always submit the explicit-feedback SYNC transfer so
  dev.pcm.%d.feedback_rate stays live as a diagnostic even when a
  capture stream is present.

Reproduced on an OKTO RESEARCH DAC8 STEREO (0x152a:0x88c5), whose
vestigial capture interface never streams; the same device plays the
44.1 kHz family correctly under Linux's snd-usb-audio.

As a side effect, this patch also fixes the sample rate bug mentioned in
the BUGS section of sound(4)'s man page, where a device needs to have
the same sample rate set for both playback and recording in order to
work properly.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295933
Assisted-By:    Claude Opus 4.8 (claude-opus-4-8)
Signed-off-by:  giacomo <delleceste@gmail.com>
MFC after:      2 weeks
Reviewed by:    christos
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2323

The fixed 128 KiB secondary buffer cap dates from stereo-sized streams.
High channel-count or high sample-width OSS streams can consume most of
that budget in one graph quantum, leaving too little room for capture
catch-up or playback headroom.

Keep 128 KiB as the low-rate floor, but derive the effective soft-ring
cap from the channel byte rate, clamped to 4 MiB. Use that per-channel
cap when resizing the soft buffer and when clamping
SNDCTL_DSP_SETFRAGMENT requests.

Also clamp SNDCTL_DSP_LOW_WATER to the current soft-buffer size so an
impossible readiness threshold cannot make poll/select wait forever.

MFC after:      3 weeks
Reviewed by:    christos
Differential Revision:  https://reviews.freebsd.org/D58064

Prevent infinite loop in uvideo_vs_negotiation() when a USB camera reports
step=0 in its continuous frame interval descriptor.
Cast fbuf_size calculation to uint64_t to avoid int overflow for large
width/height/bpp combinations.

Reported by:    emaste

Don't coerce errors to EINVAL, which isn't correct for mtx_sleep's
failure cases.

Sponsored by:   The FreeBSD Foundation

Our USB TRB buildup subroutines were previously difficult to follow. In
setup_generic_chain_sub(), the routine filled TRB packets based on the
characteristics passed by the caller and the current state (for example,
whether the TRB was the last in the TD).

However, most TRB types (except Normal TRBs) cannot be shared across TDs.
To simplify the logic, refactor xhci_setup_generic() so that TRBs are
constructed according to their transfer type, with dedicated helper
functions for each TRB type.

Sponsored by:   The FreeBSD Foundation
Assisted-by:    Claude Code (Opus 4.6, Opus 4.8(1M) and Sonet 5.0)
Differential Revision:  https://reviews.freebsd.org/D57130

When a USB HID device triggers identify,
the grandparent is usbhid on a USB hub.
Calling iicbus_get_addr() on a non-iicbus device
hits a KASSERT panic.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58432

Reviewed by:    markj
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D56311

e1000: Defer link-up notification until after TSO reset

em_automask_tso() changes the enabled TSO capabilities when the link
moves between 10/100 and 1000 Mb/s.  A running interface must be
reinitialized to apply the new capability set.  Do not publish
LINK_STATE_UP until the requested iflib reset has completed.

Replace link_active with an explicit state machine that distinguishes
the physical link, its publication to iflib, and an outstanding reset
barrier.  Preserve that barrier across a link flap with
DOWN_RESET_PENDING, and only publish DOWN if UP was previously
published.

Only request a reset for a running interface or for an initialization
while the interface is administratively up.  In other states the next
initialization will apply the capability changes, avoiding a reset
request that iflib's admin task could discard.

Reviewed by:    Faraz Vahedi <kfv@kfv.io>
Fixes:  https://cgit.freebsd.org/src/commit/?id=2ddf24f8f525 ("e1000: Automask TSO on lem(4)/em(4) 10/100 Ethernet")
MFC after:      1 week

e1000: fix 82574 MSI-X interrupt throttling

em_newitr() and the per-queue interrupt_rate sysctl both tested
que->msix to decide whether an 82574 is running in MSI-X mode.  0 is a
valid MSI-X vector so queue 0 was misclassified as legacy/MSI.

Test sc->intr_type == IFLIB_INTR_MSIX instead.  While here, index the tx
EITR read by tque->msix rather than tque->me so it matches the register
em_newitr() actually writes; the two differ once tx_num_queues exceeds
rx_num_queues.

Also seed que->itr_setting in em_initialize_receive_unit() with the rate
the hardware was just programmed with.  Otherwise an itr_setting left
over from AIM across an interface re-init makes the change detection in
em_newitr() suppress the write that would restore it, leaving the
hardware at the default rate while software believes otherwise.

Fixes:          https://cgit.freebsd.org/src/commit/?id=3e501ef89667 ("e1000: Re-add AIM")
MFC after:      3 days

e1000: fix rx accounting for multi-descriptor packets

The receive paths accumulate ri->iri_len across the descriptors making
up a packet, then add that running total to rxr->rx_bytes on every
iteration of the loop.  A packet spanning descriptors of length l1, l2
and l3 thus contributes 3*l1 + 2*l2 + l3 instead of l1 + l2 + l3.

Single descriptor packets, the common case, are accounted correctly,
so this only shows up on jumbo frames.

Add the per descriptor length instead.  iflib memsets the if_rxd_info
before each isc_rxd_pkt_get() call, so summing len gives the same total
as the final iri_len, and the frame error path that returns without
incrementing rx_packets keeps counting bytes exactly as before.

MFC after:      1 week

e1000: make AIM counter sampling coherent

Sample free-running counters by delta instead of clearing them from the
interrupt filter, which can race their producers.  Publish byte and
packet counts together at the TX and RX doorbells so each sample is
coherent.

Aggregate every TX ring assigned to the interrupt vector so unequal RX
and TX queue counts are safe.  Count RX bytes only after a frame is
accepted.

MFC after:      1 week

e1000: synchronize interrupt moderation state

Keep the saved EITR and PBA values synchronized with hardware across
reinitialization.  Correct EITR encoding, decoding, and MSI-X register
selection, and reject nonpositive fallback rates.

Treat only sub-gigabit links as sub-gigabit and apply the packet-buffer
fallback without permanently disabling AIM.

MFC after:      1 week

e1000: restore packet-size AIM

Restore the packet-size calculation introduced in a69ed8dfb381 and used
by igb(4) until the iflib conversion in f2d6ace4a684.  It derives
interrupt holdoff from average packet size, so RSS queue count does not
change its behavior.

The calculation follows the pre-iflib code.  Retain the current normal
and low-latency rate caps, and keep the current setting when an interval
has no usable sample.

Fixes:          https://cgit.freebsd.org/src/commit/?id=3e501ef89667 ("e1000: Re-add AIM")
MFC after:      1 week

e1000: count TSO wire segments in the AIM counters

The transmit paths billed one packet of ipi_len bytes per request.  For
TSO that is the whole unsegmented payload, up to 64KB, so the average
size the moderation calculation sees is not a size that appears on the
wire.

Count the segments the hardware will put on the wire and the header each
of them carries.

Non-TSO accounting is unchanged.

MFC after:      1 week

e1000: Retry transient MDIC failures on modern PCH

Some Meteor Lake and newer systems sporadically fail an MDIC PHY
transaction while the MAC and PHY clocks synchronize.  Retry twice
before reporting the transaction failure.

Disable retries around PHY interface transitions where an MDI error
is expected.  Preserve and restore the configured retry count on every
exit from those flows.

This follows DPDK commit bdca22d62ff0, extended to the PTP and NVP PCH
types.

MFC after:      2 weeks

e1000: Reconfigure modern PCH K1 clock synchronization

Meteor Lake and newer PCH generations can lose packets while the MAC
and PHY clocks synchronize.  Move K1 power-down to P1 and extend the
PHY K1 exit timeout before PHY access and after reset.

Use the longer 1 Gb/s PLL clock-gate timeout added by Linux so K1 can
remain enabled without the power penalty of disabling it.  Apply the
workaround through the newer PTP and NVP generations.

This follows DPDK commits ba54bdc79d94 and d88ef2356ecc, with the
longer exit time observed in Linux 578294b8b60d.

MFC after:      2 weeks

e1000: Compare decoded PCH LTR latencies

The LTR encoding combines a value and a nonlinear scale, so encoded
values cannot be compared directly.  Decode both the device latency and
the platform maximum before deciding whether to clamp the request.

MFC after:      2 weeks

e1000: Allow more time for PCH ULP exit

Firmware may take up to one second to unconfigure ULP, and affected
Lenovo systems have required nearly two seconds.  Allow 2.5 seconds
before treating the transition as a PHY failure.

This extends DPDK commit 7aa4c34581a5 using the field-tested bound
from Linux commit 3cf31b1a9eff.

MFC after:      2 weeks

e1000: Check PHY control register reads

Do not modify a zero-initialized PHY control value when its preceding
read failed.

MFC after:      2 weeks

igc: fix RX accounting for multi-descriptor packets

The receive path adds the running packet length to rx_bytes for every
descriptor.  A packet spanning descriptors of length l1, l2, and l3 is
therefore counted as 3*l1 + 2*l2 + l3.

Add each descriptor length once.  Single-descriptor accounting remains
unchanged.

MFC after:      1 week

igc: make AIM counter sampling coherent

Sample free-running counters by delta instead of clearing them from the
interrupt filter, which can race their producers.  Publish byte and
packet counts together at the TX and RX doorbells so each sample is
coherent.

Aggregate every TX ring assigned to the interrupt vector so unequal RX
and TX queue counts are safe.  Count RX bytes only after a frame is
accepted.

MFC after:      1 week

igc: synchronize interrupt moderation state

Keep the saved EITR value synchronized with hardware across
reinitialization.  Correct EITR encoding, decoding, and MSI-X register
selection, and reject nonpositive fallback rates.

Apply the packet-buffer fallback without permanently disabling AIM.

MFC after:      1 week

igc: use packet-size AIM

Use the packet-size calculation introduced for igb(4) in a69ed8dfb381
and retained there until the iflib conversion in f2d6ace4a684.  It
derives interrupt holdoff from average packet size, so RSS queue count
does not change its behavior.

The calculation follows the pre-iflib igb code.  Retain igc's normal and
low-latency rate caps, and keep the current setting when an interval has
no usable sample.

MFC after:      1 week

igc: count TSO wire segments in the AIM counters

The transmit path bills one packet of ipi_len bytes per request.  For
TSO that is the whole unsegmented payload, up to 64 KiB, rather than a
packet size that appears on the wire.

Count the segments the hardware emits and the header carried by each
segment.  Non-TSO accounting is unchanged.

MFC after:      1 week

No functional change intended.

Sponsored by:   The FreeBSD Foundation
MFC after:      1 week

Previously, multiple frames of xfers are split into many tds. In the
refactor process, we forget to consider this. The td builder is already
allocate with enough numbers of tds. What we need to do is to fill the
normal trbs for muiltiple tds when building trbs.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297053
Tested by:      phk, oleglelchuk@gmail.com
Fixes:  https://cgit.freebsd.org/src/commit/?id=e0b235ecd4fa ("xhci: Refactor xhci_generic_setup code")
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58465

It is better to propagate it to pcm_register(), and later to the device
drivers, than to simply ignore it and return ENXIO.

Sponsored by:   The FreeBSD Foundation
MFC after:      1 week

Sponsored by: The FreeBSD Foundation
MFC after:      1 week

In align_abort() and tag_check_abort(), if we got a fault while in kernel,
do not panic if a fault handler has been provided. We may get such a fault
when trying to read or write userland data, it can at least happen with
_umtx_op() if an unaligned pointer is provided. Instead, just let the
fault handler deal with it.

MFC After: 1 week
Approved by: andrew
Differential Revision: https://reviews.freebsd.org/D58426

specialreg.h is the tree's MSR registry and already carries the Intel
RAPL group. Add the AMD RAPL package/core energy and unit MSRs here so
the hwpmc RAPL class can reference them without a private driver copy.
Use the names Linux's msr-index.h gives these registers.

Reviewed by:    mhorne, adrian, Ali Mashtizadeh <ali@mashtizadeh.com>
MFC after:      3 days
Sponsored by:   AMD
Differential Revision:  https://reviews.freebsd.org/D58027

Add hwpmc_rapl.c/.h implementing PMC_CLASS_RAPL, a read-only
system-scope class modeled on TSC and wired into x86 AMD and Intel MD
init. A per-vendor MSR table covers AMD/Hygon and Intel; energy is
reported in microjoules, with the Intel server 2^-16 J DRAM unit
handled and 32-bit wraps recovered into a 64-bit accumulator.

The overflow guard follows the PMC lifetime: armed on the first
allocated PMC, callout_drain()d on the last release, and each tick only
rendezvouses CPUs holding one. Per-CPU spin locks guard the accumulator
against torn reads on i386. PMC_CAP_DOMWIDE lets pmcstat(8) allocate
one counter per NUMA domain instead of per CPU.

Reviewed by:    mhorne, Ali Mashtizadeh <ali@mashtizadeh.com>
Sponsored by:   AMD
Differential Revision:  https://reviews.freebsd.org/D58028

Register PMC_CLASS_RAPL in libpmc: event table, allocator, class-table
descriptor, and the event-name/class-listing lookups, all x86-guarded
and modeled on the TSC class. Energy events are read-only and
unqualified. The class prefix (RAPL-) supplies the friendly spelling, so
pmcstat -S rapl-energy-pkg resolves to the canonical ENERGY_PKG event.

Add a pmc.rapl.3 manual page documenting the events, counter scope, the
microjoule unit and wrap handling, and the NUMA/package domain mapping;
link it from pmc.3.

Reviewed by:    mhorne
Discussed with: Ali Mashtizadeh <ali@mashtizadeh.com>
Sponsored by:   AMD
Differential Revision:  https://reviews.freebsd.org/D58029

Neither of these options are checked in the file and cdefs.h should not
be included explicitly. No functional change.

Sponsored by:   The FreeBSD Foundation

amd_allocate_pmc() chose the pmu-events code path whenever pmc_cpuid was
non-empty, and rejected any allocation lacking PMC_F_EV_PMU.
But pmc_cpuid is set for every AMD CPU, while the pmu-events tables only cover
Zen and later.
On older families (K8, Bobcat, Jaguar/16h, Bulldozer) libpmc finds no
pmu-events entry and falls back to the legacy path, which never sets
PMC_F_EV_PMU.

Reviewed by:    mhorne
Approved by:    mhorne
MFC after:      1 week
MFC to:         stable/14, stable/15
Sponsored by:   Netflix
Differential Revision:  https://reviews.freebsd.org/D58468

{em,igb}_determine_rsstype() mapped only the TCP and bare-IP RSS descriptor
types; the UDP types returned M_HASHTYPE_NONE.
The hardware does hash UDP, but with a NONE hashtype iflib skips its
flowid-based TX queue spread, so all forwarded UDP egressed on a single queue
and serialized transmit on one core.

Add the three UDP cases (IPV4_UDP, IPV6_UDP, IPV6_UDP_EX) so egress spreads
across all TX queues.

Reviewed by:    kbowling, gallatin
Approved by:    kbowling
MFC after:      1 week
MFC to:         stable/14, stable/15
Sponsored by:   Netflix
Differential Revision:  https://reviews.freebsd.org/D58513

Add a QUIRK_EMPTY_NAMESPACE_CHANGED_LOG quirk which indicates that the
nvme controller may not properly populate the namespace-changed log
page.  If we receive a NVME_LOG_CHANGED_NAMESPACE page for a device
with this quirk and the page is empty, probe all of the namespaces
rather than none of them.

Reviewed by:    imp
MFC after:      1 week
Sponsored by:   Amazon
Differential Revision:  https://reviews.freebsd.org/D58231

This controller exhibits QUIRK_EMPTY_NAMESPACE_CHANGED_LOG behaviour.

A bug report has been filed with the vendor.

Reviewed by:    imp
MFC after:      1 week
Sponsored by:   Amazon
Differential Revision:  https://reviews.freebsd.org/D58232

In particular, handle authentication errors due to bad MACs when
decrypting packets.

Since the current dispatch code assumes synchronous OCF sessions by
design, explicitly reject any created OCF session that is not
synchronous.  Software sessions are always synchronous in practice, so
this should be a nop.

Approved by:    so
Security:       FreeBSD-SA-26:52.if_wg
Security:       CVE-2026-58085
Reviewed by:    markj
Sponsored by:   Chelsio Communications

Intel Apollo Lake SDXC controller reports a Slot Type of
"Embedded Slot for One Device" in SDHCI_CAPABILITIES bits, even
when the slot is a removable card reader.
This caused 48 timeouts before the boot sequence resumed.

Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D58467

Update the shared e1000 PF/VF mailbox interfaces for an in-tree igb
SR-IOV implementation.

Intel FreeBSD igb-2.5.31 and DPDK provide the older PF/VF mailbox
baseline.

The retained PF mailbox read and explicit unlock operation follow a
simple Linux igb parameter addition to make PF mailbox acquisition
nonblocking so the driver can retry outside the shared primitive.

Treating a CTS-less E1000_PF_CONTROL_MSG as a reset follows DPDK.

Sponsored by:   BBOX.io

Use each ring's physical queue index for initialization, MSI-X routing,
register dumps, sysctls, and debug output instead of assuming that its
logical array index is also its hardware index.

This is a no-op for the normal queue layout.  A later SR-IOV change
moves the PF ring to hardware queue num_vfs, so its hardware ID then
differs from logical queue zero.

Sponsored by:   BBOX.io

A non-zero VF device number does not always require ARI. The Intel
82576 and I350 [1] explicitly support a non-ARI layout that places VFs
on the next bus.

Check every requested VF RID and reject a non-zero device only when it
is on the PF bus. This retains the ARI guard for invalid same-bus
layouts while permitting the documented second-bus layout.

[1] Intel I350 Datasheet, sections 7.8.2.6.1.2, 9.6.4.6

Sponsored by:   BBOX.io

pci_iov_config() programs NumVFs before validating the final VF RID
layout and allocating all generic resources. A subsequent error ran the
driver uninit callback but left the hardware NumVFs register programmed
while the software VF count returned to zero.

Clear NumVFs in the error path after the driver uninit callback,
matching normal SR-IOV teardown ordering. This prevents stale hardware
state after a failed configuration and permits a clean retry.

MFC after:      1 week
Sponsored by:   BBOX.io

Register the 82576 and I350 VF PCI IDs under a separate igbv driver
while continuing to share the igb datapath implementation.

Follow the ixv driver split and give the VF context IFLIB_IS_VF so
iflib does not apply the PF SR-IOV detach guard to a child VF. Program
VTIVAR_MISC in the VF low byte so mailbox and reset notifications reach
the VF admin vector.

The split will become increasingly obvious as bug fixes land, trying to
bias everything with if (sc->vf_ifp) everywhere is error prone in two
directions.

This breaks existing naming/configurations and cannot be MFCed as-is.
I have no plans of adapting it to prior branches at the moment but it
may be possible.

Relnotes:       yes
Sponsored by:   BBOX.io

I350 loopback receive descriptors report VLAN tags byte-swapped for
both PFs and VFs. The receive path handled the PF device types but
omitted e1000_vfadapt_i350, causing an admitted VF VLAN packet to be
delivered untagged to the VF parent.

Include the I350 VF type in the existing correction. This matches the
dedicated IGB_RXQ_FLAG_LB_BSWAP_VLAN handling in DPDK igbvf.

MFC after:      1 week
Sponsored by:   BBOX.io

The register-dump sysctl is installed before iflib allocates the queue
arrays and remains visible while they are freed. Return ENXIO outside
the queue lifetime instead of dereferencing a NULL or stale array.

Sponsored by:   BBOX.io

Add the PCI IOV schema and PF control plane for up to seven VFs with
one hardware queue per pool. Implement VF mailbox handling, MAC and
VLAN assignment, multicast filtering, promiscuity policy,
anti-spoofing, malicious-driver recovery, reset replay, and queue
lifecycle management.

The basic SR-IOV and VMDq PF implementation follows DPDK Intel e1000
code, including PF pool selection, one queue per pool, mailbox
dispatch, and VF enablement. Intel FreeBSD igb-2.5.31 supplies the
older driver baseline. Linux igb and the Intel SDMs clear up lifecycle,
isolation, reset, and family-specific details absent from DPDK.

Enabling IOV requires the PF to attach with one TX and RX queue.
Systems whose defaults select RSS queues must set the documented iflib
queue override tunables before attach.

Only 82576 and I350 support SR-IOV in silicon.  The series has been
extensively tested on I350, including thowing boundaries at the PCI BAR
that shipping drivers will never.  Still, think carefully before
reaching for this in critical environments.

Relnotes:       yes
Sponsored by:   BBOX.io

Disable each igb-class transmit and receive queue and flush before
changing its descriptor-ring registers. Restore the head and tail
indices that Intel documents as surviving a VF reset.

Use the igb queue-enable control instead of programming legacy TXDCTL
granularity, low-water, and reserved bits that do not belong to the
82575 and later.

Sponsored by:   BBOX.io
MFC after:      1 week

igbv: Isolate VF policy and validate its registers

Give igb virtual functions a separate ifdi method table and move
VF-specific attach, reset, queue, interrupt, and diagnostic policy to
if_igbv.c. Keep shared descriptor-ring mechanisms in if_em.c.

Derive VF identity from IFLIB_IS_VF and assert that hardware
identification agrees. Under INVARIANTS, validate normal VF CSR
accesses against the sparse 82576 and I350 VF register maps.

Stop shared setup from accessing PF-only controls. Require MSI-X and
defer VF sysctls until attach succeeds so failed attachment cannot
leave handlers pointing at freed driver state.

Advertise only VF capabilities, run adaptive moderation without the
PF receive-buffer guard, enable SRRCTL.DROP_EN, and provide a VF-safe
diagnostic register view.

The moved implementation is the existing FreeBSD code. Register model
was cross-checked against the Intel datasheets and other Intel drivers.

Sponsored by:   BBOX.io

igbv: Improve VF mailbox and status behavior

Treat VF media as fixed 1000baseT full duplex and report PF not ready
and generated MAC fallback states during attach.

After a successful reset handshake, reconcile a PF rejected MAC back
into the ifnet. If the PF is unavailable, defer MAC, multicast, VLAN,
LPE, and promiscuity replay until CTS is restored. Track a rejected
VLAN removal separately so leaked traffic remains tagged until reset
proves that the stale hardware filter is gone.

Baseline VF counters at attach, collect the four loopback packet and
octet counters with rollover-safe deltas, and account software RX
checksum offload results. Preserve accumulated statistics across PF
resets by rebasing the raw hardware counters, and sample them while
physical link is down because VF loopback can remain active.

Retain the 82576 VFMPRC hardware statistic, but do not read it on I350
VFs because specification update errata 31 says it is unavailable.

Clear PF owned flow control state and reset a link down VF when queued
transmit descriptors must be flushed.

Sponsored by:   BBOX.io

igbv: Support secondary unicast filters

Support the Linux igbvf secondary-MAC mailbox subprotocol, used by
Linux guests running MacVTap.

Replay up to three non-primary unicast addresses after reset and
whenever the address list changes, subject to PF allow-set-mac policy.

Sponsored by:   BBOX.io

igb: Stop writing the legacy TADV register

TADV is an em-class interrupt delay register and is absent from the
82575 and later register model. The igb attach path does not expose or
initialize that control, but transmit initialization still wrote its
zero valued storage into a reserved queue-window offset.

Apply the same igb_mac_min boundary already used for TIDV and the
absolute-delay sysctls.

MFC after:      1 week
Sponsored by:   BBOX.io

igb: Update only changed IOV multicast hashes

Build the aggregate PF/VF multicast bitmap in software and compare it
with the e1000 MTA shadow. Write only registers whose desired value
changed, while forcing a complete write after PF reset invalidates the
hardware table.

This bounds alternating VF multicast updates without NACKing them.
Linux igbvf and DPDK ignore multicast reply status, so a command-rate
limiter could otherwise acknowledge configuration while leaving
hardware state stale.

Sponsored by:   BBOX.io

igb: Update only changed IOV VLAN filters

Keep the full VFTA/VLVF software recomputation and clear-map-set
ordering, but compare each phase against the authoritative old value.
Write only VFTA words and VLVF slots whose effective contents change.

I350 uses its software VFTA shadow because erratum 20 makes live reads
unreliable; an invalid shadow forces a complete clear before sparse
restoration. 82576 continues to diff against live VFTA reads.

Add SDT probes for every logical write phase and the final software
images so hardware tests can verify exact elision counts. On my I350
DUT, the old full table path averaged 819 us across 31 VLAN removals
versus about 79 us for the PF statistics sweep.

Sponsored by:   BBOX.io

igb: Rate-limit VF VLAN rebuild requests

Give each VF a burst of 64 VLAN additions and refill it at eight
additions per second. Removals remain unrestricted, idempotent requests
consume nothing, and trusted PF-wide initialization replenishes the
burst while guest resets do not.  Checks VLVF capacity before charging
a token.

Do not apply this policy to multicast requests because Linux igbvf and
DPDK ignore their reply status; aggregate MTA write elision bounds
those updates instead.

Sponsored by:   BBOX.io

Mailbox and link interrupts share iflib admin service with the periodic
timer. Mark timer-driven passes explicitly and run the hardware
statistics sweep only for those samples instead of repeating 66 PF MMIO
reads for every VF mailbox message.

DTrace on the I350 DUT measured the PF sweep at about 79 us on average.
The normal hz/2 timer continues to extend clear-on-read counters
safely; exported counters may trail hardware by up to 500 ms.

Sponsored by:   BBOX.io

A PF mailbox NACK does not distinguish the SR-IOV VLAN request rate
limit from permanent VLVF exhaustion. Preserve desired VLAN membership
and retry four additions per 500 ms timer tick, matching the PF
sustained allowance.

Bound the whole recovery batch to eight seconds from its first failure
and consolidate restore diagnostics, so a full table cannot create a
permanent mailbox poller or repeated per-VID log bursts.

Sponsored by:   BBOX.io

Pass the VF generation through the CSR accessors so the validator can
distinguish the sparse 82576 and I350 register maps.

Admit the queue-zero RXCTRL, TXCTRL, TDWBAL, TDWBAH, and
VFPSRTYPE registers exposed by both families.

82576 exposes VFMPRC at 0xf3c.  I350 erratum 31 makes its
corrected 0xf38 address inaccessible to a VF, so reject both I350
spellings while retaining read access on 82576.

Sponsored by:   BBOX.io

82576 and I350 VFLR leave the VF queue configuration unchanged.  A VF
can program transmit head write-back and leave its DMA destination for
a later VF owner; mainstream VF drivers do not overwrite TDWBAL/H.

Disable every receive and transmit queue assigned to the VF, wait for
the enable bits to clear, then clear SRRCTL, PSRTYPE, RXCTRL, TXCTRL,
and TDWBAL/H.  Spin briefly for the normal transition, then sleep at
100 microsecond intervals with an approximately 1 ms bound.  This
prevents a VF that keeps asserting QUEUE_ENABLE from busy-waiting the
PF context lock for 10 ms.

If a queue does not quiesce, leave the VF disabled and NACK its reset
rather than programming an active queue.  Rate-limit this diagnostic
independently from mailbox and malicious-driver notifications.

I350 maps pool n to queue n.  82576 assigns physical queues n and n+8
to VF n, so sanitize both queues while clearing per-pool PSRTYPE once.
An incoming VF initializes its active ring base, head, and tail while
enabling each queue.

This is also required by malicious-driver recovery, which deliberately
does not assert VTCTRL.RST because doing so would discard the VF's
admin-vector routing before the PF can notify it.

This implements Software Clarification 3 from the 82576 and I350
specification updates.

Sponsored by:   BBOX.io

82576 and I350 VFLR leave queue configuration unchanged.  A previous
VF owner can therefore leave a transmit head-writeback DMA destination
and other queue policy for the next guest.

After each reset attempt, disable all exposed VF queues and wait for
their enable bits to clear before clearing SRRCTL, VFPSRTYPE, RXCTRL,
TXCTRL, and TDWBAL/H.  Spin briefly and then sleep until the bounded
queue-disable deadline.

iflib cannot report initialization failure and marks an interface
running after its init callback returns.  On sanitation failure, keep
interrupts disabled and use the deferred admin task to clear RUNNING.
Retry after 100 and 500 ms; after three total failures, leave the
interface down until another administrative initialization starts a
new bounded attempt set.

igbv uses queue zero on both families, but 82576 exposes a second VF
queue whose retained state must also be cleared.  Extend the INVARIANTS
register validator for only those queue-one CSRs and only on 82576.

This implements the VF side of Software Clarification 3 from the 82576
and I350 specification updates.  It also means an igbv guest does not
depend on its PF to sanitize a previous VF owner's state.

Sponsored by:   BBOX.io

amd64: do not allow to set reserved bits in MXCSR for ptrace(PT_SETFPREGS)

Also do not mask bits in the mxcsr_mask.  It is ignored by FRSTOR/XRSTOR.

Reported by:    markj
Reviewed by:    jhb, markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58548

ptrace: Propagate errors from set_fpregs()

Otherwise ptrace(PT_SETREGSET) will not return errors to userspace.

Fixes:          https://cgit.freebsd.org/src/commit/?id=cef05c5a62ba ("amd64: do not allow to set reserved bits in MXCSR for ptrace(PT_SETFPREGS)")
Reviewed by:    kib
Differential Revision:  https://reviews.freebsd.org/D58577

Reported by:  jhb
Reviewed by:    jhb, jrtc27
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58550

Some conventional PCI e1000 configurations hang when given DMA
addresses above 4 GB, particularly on systems using AMD
HyperTransport-to-PCI bridges.  Linux has restricted e1000 to DMA32 in
PCI mode since 2011 for the same failure class in commit
e508be174ad36b0cf9b324cd04978c2b13c21502.

Set iflib's DMA width after determining the negotiated bus type.  This
covers descriptor and packet-buffer mappings while preserving 64-bit
DMA for PCI-X and PCIe devices and providing a conditional tunable.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297064
Reported by:    Alexander Leidinger <netchild@FreeBSD.org>
Tested by:      Alexander Leidinger <netchild@FreeBSD.org>
MFC after:      1 week

Add register/bit definitions for the L1 PM substates capability
(PCIZ_L1PM) to pcireg.h.

Signed-off-by: Michael Adler <madler@tapil.com>

MFC after:      1 week
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2318

I226 parts advertise support for the PCIe L1.2 link substate, but a
hardware erratum makes the exit latency from that low-power state
longer than the packet buffer can absorb under load. This stalls the
inbound packet stream. Disabling ASPM system-wide (BIOS or OS ASPM
policy) does not fix it. The L1.2 enable bit must be cleared directly
in the device's own PCIe L1 PM extended capability.

Add igc_is_device_id_i226() to identify affected parts and
igc_disable_broken_aspm_l1_2() to clear the ASPM L1.2 enable bit
on attach and after resume, since PCIe config space can be
reset across a suspend/resume cycle.

Adapted from the Linux igc driver:

  0325143b59c6 igc: disable L1.2 PCI-E link substate to avoid
               performance issue
  1468c1f97cf3 igc: fix disabling L1.2 PCI-E link substate on I226
               on init

Signed-off-by: Michael Adler <madler@tapil.com>

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=279245
Reviewed by:    Jim Thompson
MFC after:      1 week
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2318

ixgbe: isolate VF reset state

IXGBE_VF_INDEX() selects a 32-VF register bank.  PFMBMEM() selects
one mailbox per VF, while ixgbe_toggle_txdctl() calculates queue
offsets from a VF number.  Passing the bank index aliases VF1-31 to
VF0 and VF32-63 to VF1.  Resetting one VF can therefore clear the peer
mailbox and leave its transmit queues disabled.

The VF raises its reset event before posting its mailbox request.  The
PF checks reset events before mailbox messages.  If both are pending,
clearing PFMBMEM during generic reset handling can erase the request
before ixgbe_read_mbx() consumes it.  Clear the mailbox only from the
reset-message handler after the request has been read.

Use the VF number for queue toggling and document that API contract.

MFC after:      1 week

ixgbe: respect peer mailbox ownership

A VF currently treats an existing VFU bit as a successful acquisition,
while the PF checks its own PFU bit before claiming the mailbox.  Check
both the local and peer ownership bits before setting local ownership.
This prevents same-side callers from sharing the mailbox and avoids an
acquisition attempt while the peer owns it.

VFLR does not clear VFMAILBOX.VFU.  Clear stale VF ownership and cached
mailbox status after the reset indication settles and before sending the
reset request, so the ownership check cannot strand a reinitialized VF.

Adapt only the live ownership checks from Intel ix 3.4.39.  Do not import
its upgraded-mailbox changes, which are not active in FreeBSD.

Obtained from:  Intel ix 3.4.39
MFC after:      1 week

ixgbe: fail fast on VF-held PF mailboxes

The active PF mailbox operations use the legacy helpers.  The mailbox API
import changed check_for_msg into a read-only probe and added up to 2,000
500-microsecond lock retries.  If a VF leaves VFU set, the PF cannot acquire
the lock, busy-waits for up to one second, and leaves VFREQ pending so the
delay can repeat.

Give the legacy checker its old consume-on-check behavior so a failed read
does not leave VFREQ asserted.  If VFU is already set, fail immediately
instead of retrying, while preserving retries for PF-side contention.  Do
not force RVFU, which would discard peer transaction state.

MFC after:      1 week

ixgbe: enforce configured VF anti-spoofing

The SR-IOV schema advertises MAC anti-spoofing and enables it by
default, but the VF configuration was never consumed and the hardware
policy remained disabled.

Record the configured policy and apply MAC and VLAN anti-spoofing
throughout VF initialization and reset. On X550-family devices, also
protect the LLDP and flow-control Ethertypes and enable per-VF
spoof-event accounting. Remove the driver-owned state during SR-IOV
teardown.

Adapt the anti-spoof configuration lifecycle used by igb(4) in
a2ed165f0049 to the ixgbe hardware controls.

MFC after:      1 week
Relnotes:       yes

ixgbe: preserve VLAN ownership with SR-IOV

The VF VLAN capability is checked but never granted, and no SR-IOV
configuration property exposes the existing default-VLAN support. PF
VLAN updates also replace VFTA registers from a PF-only shadow, erasing
live VF filters.

Expose access VLAN and trunk policy through the IOV schema. Track each
VF VLAN as desired state, restore the administrative VLAN after reset,
and use the native VLVF helper for incremental PF and VF ownership
changes.

Keep VLAN filtering enabled while SR-IOV is active. When PF hardware
filtering is disabled, admit every VLAN to the PF without bypassing
per-pool VF isolation. Reconstruct VLVF and the shared VFTA from PF and
VF desired state after reset or a filtering-mode transition, and
restore PF-only state on teardown.

When the last VF leaves a VLAN still owned by the PF, free its VLVF
slot while retaining the shared VFTA bit. This prevents a trunk VF from
exhausting the 64-entry VLVF table by cycling VLAN memberships.

Adapt the VLAN ownership model introduced for igb(4) in a2ed165f0049 to
ixgbe's native VLVF machinery.

Match Linux receive semantics by exposing a stripped VLAN tag only
when that VID was registered by the VF.  A PF-assigned port VLAN is an
administrative tag and must be delivered to the VF as untagged traffic;
otherwise the stack dispatches it to a nonexistent VLAN interface and
access-VLAN receive traffic is blackholed.

MFC after:      1 week
Relnotes:       yes

ixgbe: Preserve priority-tagged traffic with SR-IOV

VID 0 carries only 802.1p priority and does not identify VLAN
membership. Keep VFTA bit zero in the persistent PF shadow table so
reset and SR-IOV replay admit priority-tagged frames while VLAN
filtering is enabled.

In virtualization mode, also reserve VLVF slot zero and restore PF and
eligible VF pool memberships. A VFTA hit alone admits the tag globally
but does not deliver it to the correct pools.

This matches the priority-tag treatment in em/igb.

MFC after:      1 week

ixgbe: enforce VF promiscuity and multicast policy

The allow-promisc IOV property is advertised but ignored, and the PF
rejects the xcast request used by modern VFs. Negotiate mailbox APIs
1.2 and 1.3, implement pool-scoped xcast modes, and require
allow-promisc for requested all-multicast or unicast-promiscuous modes.

The VF mailbox can carry only 30 multicast hashes. When ixv has a
larger list, request the API 1.2 all-multicast xcast mode instead of
extending the legacy SET_MULTICAST message. The PF grants that fallback
only to VFs configured with allow-promisc; otherwise ixv reports that
only the first 30 addresses are active.

Reset xcast state with the VF and have ixv replay the mode implied by
its interface flags after multicast updates.

Follow DPDK's ixgbe API 1.2/1.3 xcast contract, with allow-promisc
policy adapted from igb(4) in a2ed165f0049.

MFC after:      1 week
Relnotes:       yes

ixgbe: implement VF secondary MAC filters

The PF advertises the legacy SET_MACVLAN mailbox request but always
rejects it. The request installs secondary unicast addresses.

Allocate an owned RAR pool for VF secondary addresses, reserve low
entries for PF filters, and place VF-primary addresses at the top of
the usable RAR range. Reject address collisions and cap each VF at
three secondary filters so one guest cannot exhaust the shared table.

Clear secondary filters on VF or PF reset and on SR-IOV teardown. This
hardware can anti-spoof only the VF primary source address. Reject
secondary filters while MAC anti-spoofing is configured, so installing
them requires an explicit administrative policy choice. Report optional
filter-table allocation failure without disabling SR-IOV.

Adapt the owned-RAR allocation and reset-cleanup model from igb(4) in
a2ed165f0049 to DPDK's ixgbe SET_MACVLAN mailbox semantics.

MFC after:      1 week
Relnotes:       yes

ixgbe: Preserve VF jumbo frame size across PF resets

sc->max_frame_size represents the largest frame requested by the PF or
an active VF.  The MTU callback replaces it with the PF frame size, so
a subsequent reinitialization can program MHADD below an active VF's
jumbo-frame request.

Recompute the aggregate before hardware initialization and use it when
programming MHADD.  Recompute after each VF LPE request as well, so a
reduced request can lower the hardware limit when no other function
needs the previous value.

MFC after:      2 weeks

ixgbe: Restore missed packet accounting

missed_rx and total_missed_rx are never populated.  As a result, the
GPRC erratum workaround does not remove missed packets and iqdrops
always remains zero.  The rx_missed_packets sysctl and input-error total
also expose only MPC bank zero.

Read and accumulate all eight MPC banks.  Use the interval total to
correct GPRC and the cumulative total for iqdrops, input errors, and the
aggregate sysctl.  This matches DPDK's coverage of the hardware banks.

MFC after:      2 weeks

ixgbe: Validate EEPROM checksum section bounds

The generic checksum walker trusts NVM section pointers and lengths and
iterates with a 16-bit index.  A corrupt section that crosses the end of
the EEPROM can wrap the index and leave the driver in an effectively
unbounded read loop during attach.

Validate each non-empty section against the discovered EEPROM word size
before reading it, and use widened arithmetic for the inclusive end and
iterator.

MFC after:      2 weeks

ixgbe: Compare flow control against requested mode

The flow-control sysctl represents the configured policy, while
current_mode is the mode negotiated with the link partner.  Comparing a
new request with current_mode can needlessly reprogram an unchanged
policy or skip a requested policy change that happens to match the
current negotiation result.

Compare with requested_mode before deciding that no update is needed.

MFC after:      2 weeks

if_foreach_llmaddr() adds each callback return value to its running
count.  Returning the incremented count made the address indices grow
as 0, 1, 3, 7, and so on, eventually writing beyond the multicast
address array.

Return one address per callback and stop copying when the array is
full, matching the ixv-1.6.12 driver.

Fixes:          https://cgit.freebsd.org/src/commit/?id=ff06a8dbb677 ("Mechanically convert ixgbe(4) to IfAPI")
MFC after:      1 week

DPDK commit message

net/ixgbe/base: add missing buffer copy for ACI

Add the missing buffer copy in ixgbe_aci_send_cmd().

The retry path saves the original descriptor and allocates storage for
the command buffer so both can be restored before another attempt.  It
did not copy the original command buffer into that storage.

Fixes:          https://cgit.freebsd.org/src/commit/?id=25b48e569f2f
Cc: stable@dpdk.org

Signed-off-by: Dan Nowlin <dan.nowlin@intel.com>
Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (37239792b0)
MFC after:      1 week

DPDK commit message

net/ixgbe: fix flow control frame byte adjustment

LXONTXC and LXOFFTXC are 32-bit counters for transmitted XON and XOFF
packets.  Their deltas are summed and used to adjust the transmitted
packet and byte counters.

Perform the addition in 64 bits so it cannot wrap before the result is
used for the byte adjustment.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Fixes:          https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Daniil Iskhakov <dish@amicon.ru>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (bdf8608559)
MFC after:      1 week

DPDK commit message

net/ixgbe/base: fix unchecked return value

Check the return value from ixgbe_read_eeprom() before using the
control word to configure link disable during D3.

Fixes:          https://cgit.freebsd.org/src/commit/?id=b7ad3713b958 ("ixgbe/base: allow to disable link on D3")
Cc: stable@dpdk.org

Signed-off-by: Barbara Skobiej <barbara.skobiej@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (eb3684b191)
MFC after:      1 week

FreeBSD's I2C helper already retries failed transactions.  Limit this
new outer loop to successful reads with an invalid identifier so that
retry budget is not multiplied.

DPDK commit message

net/ixgbe: retry misbehaving SFP read

Some XGS-PON SFPs ACK I2C reads and return uninitialized data while
their microcontroller boots.  A bogus identifier can cause an otherwise
working module to be marked unsupported.

Retry the identifier read several times, checking for both successful
I2C completion and a valid SFP identifier.

Signed-off-by: Stephen Douthit <stephend@silicom-usa.com>
Signed-off-by: Jeff Daly <jeffd@silicom-usa.com>
Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>

Obtained from:  DPDK (774263bb4e)
MFC after:      1 week

ixgbe: fix unaligned access in ixgbe_update_flash_X550()

ixgbe_host_interface_command() treats its buffer as a u32 array.  The
local union contained only byte-sized fields, giving it one-byte stack
alignment and allowing unaligned accesses on strict-align systems.

Add a u32 member to the union to provide the required alignment and
pass that member to ixgbe_host_interface_command().

No functional change is expected on x86.

Obtained from:  Intel ix 3.4.39
MFC after:      1 week

ixgbe: avoid signed overflow in pause time calculation

pause_time is promoted to signed int before multiplication.  Its default
value of 65535 multiplied by 65537 exceeds INT_MAX and triggers UBSAN,
even though the result is assigned to a u32.

Make the multiplier unsigned so the calculation has the intended u32
semantics.  Linux commit 3b70683fc4d6 reported the failure in the generic
path and used the same mechanical correction.  The 82598-specific flow
control operation contains the identical expression, so correct it as well.

MFC after:      1 week

ixgbe: reject VF requests before CTS

A VF that sends a non-reset request before completing reset negotiation
has not received CTS.  The PF ignores the request but currently reports
success, leaving the VF with a false view of the programmed state.

Return failure for the ignored request.  This restores the behavior lost
when the mailbox helpers were renamed.

Fixes:          https://cgit.freebsd.org/src/commit/?id=36c516b31136 ("ixgbe: update if_sriov to use the new mailbox apis")
MFC after:      1 week

ixgbe: check negotiated API for VF queue query

The GET_QUEUES handler switches on msg[0], which contains the mailbox
command rather than the negotiated API version.  It therefore cannot
reject API 1.0 or an unnegotiated VF as intended.

Switch on the API version stored for the VF.

MFC after:      1 week

ixgbe: complete PF cleanup after VF FLR

The 82599, X540, and X550 documentation identifies VF registers which
retain state across VFLR and must be reconfigured before a VF is reused.
The VF reset path already initializes its queue-owned registers, but the
PF only cleared VF mailbox memory and transmit head write-back addresses
after a cooperative mailbox reset.  A bare hardware VFLR therefore left
both behind on affected devices.

Move TDWBA cleanup into the common reset path.  Clear CTS when VFLR
invalidates the mailbox session, and accept only VF_RESET during the
reset pass before restoring VF traffic.

Clear VFMBMEM through the PFU/VFU semaphore.  Recheck VFREQ while
holding PFU so a reset event cannot erase a request posted between the
initial mailbox check and the clear.  Dispatch an already-read message
even if the residual clear fails, but keep cleanup pending until a
synchronized clear succeeds.  Retry cleanup in the same admin pass
after a failed message read or clear.

The 82599 also retains VFMAILBOX.VFU across VFLR.  Leave a VF-owned
mailbox intact initially so a live post-reset writer can finish.  Retry
cleanup from the admin timer and, after a two-second grace period, use
PFMAILBOX.RVFU only when VFU remains set and no request has been posted.
Clear the mailbox under PFU afterward.  This recovers an abandoned
pre-reset owner without sleeping under the iflib context lock or
immediately stealing from a new reset request.

Suppress mailbox dispatch once iflib has cleared IFF_DRV_RUNNING so a
pending reset request cannot re-enable VF traffic inside the PF stop
path.  Periodically sample aggregate VFREQ, VFACK, and VFLR registers,
masked to active VFs, so work suppressed across a stop/restart and a
bare 82599 VFLR without EICR_MAILBOX are both discovered without another
interrupt edge.

MFC after:      2 weeks

ixgbe: recover from X550 malicious-driver events

The shared X550 code provides malicious-driver detection, event
decoding, and per-pool recovery operations, but the PF never enables or
services them. A malformed VF descriptor can therefore go undetected
and avoid the per-pool recovery path supplied by the MAC.

Configure IOV state while VF DMA remains disabled, then enable MDD and
activate the VFs only after PF queue initialization is complete. On an
MDD event, withdraw mailbox CTS and gate the VF pool through PFVFTE and
PFVFRE. Retain the per-queue WQBR blocks until the VF enters a new reset
epoch; PFVFTE can still permit descriptor fetches into the internal
queue, so releasing WQBR early would allow a hostile VF to retrigger
MDD before it resets.

Send the non-CTS reset notification after servicing the VF mailbox.
Let a posted VF request win mailbox arbitration, defer notification if
the pass produced a response, and retry failed notifications from the
periodic admin pass. Poll WQBR so recovery does not depend on another
mailbox interrupt edge, while suppressing already-fenced pools. Latch a
PF reset request until the next hardware initialization.

The X550 datasheet defines every bit of WQBR_RX and WQBR_TX as a queue
bit, so an all-ones value is valid. Reject it only when IXGBE_STATUS,
which has reserved-zero bits, also reads as all ones and confirms dead
MMIO.

Temporarily disable MDD around live multiqueue SRRCTL drop-mode updates,
which hardware otherwise reports as queue-context changes. Serialize
that window with the iflib context lock and resample pending work after
MDD is restored.

Apply the per-pool recovery model used by igb(4) in a2ed165f0049 to the
existing DPDK-derived X550 hooks. The same register interface is
documented for X552 and X553, so cover the entire X550 family. Document
that VF traffic remains disabled until the reset handshake completes.

MFC after:      2 weeks
Relnotes:       yes

ixgbe: force receive drops on every VF queue

PFQDE is indexed by absolute receive queue, but the driver programs one
index per VF. Only the first quarter or half of the VF queues therefore
have queue-drop isolation, depending on the virtualization mode. The
flow-control path can also clear those bits even though SR-IOV requires
them independently of the PF pause policy.

Program every queue in a VF pool before enabling receive for that VF.
For an X550-family VF with an administrative port VLAN, also hide the
VLAN tag as the hardware requires. Keep PF flow-control changes
confined to the PF SRRCTL registers, and clear the VF queue settings
when SR-IOV is torn down and the queues can be reassigned to the PF.

MFC after:      2 weeks

ixgbe: Avoid a signed shift while assembling the PBA number

The EEPROM word is promoted to signed int before the left shift when
the cast is applied to the complete expression.  Cast the word first so
all 16-bit values are shifted as unsigned data.

This is the ixgbe counterpart of the e1000 correction imported from
DPDK commit b932270c66.

MFC after:      2 weeks

ixgbe: Avoid signed overflow in LED register masks

LED index three shifts the blink bit into bit 31.  Convert the base to
the register width before shifting so the operation is unsigned.

This is the ixgbe counterpart of the e1000 correction imported from
DPDK commit 214cb0d7f1.

MFC after:      2 weeks

ixgbe: Use unsigned register bitmap shifts

VLAN, VMDq, and VF reset bit indices can reach 31.  Use unsigned
values when constructing their 32-bit register masks so the shifts do
not operate on signed integers.

MFC after:      2 weeks

Clear ROMPE for an empty list and enable it only for a nonempty list.
FreeBSD already clears ROMPE when resetting a VF, so that part of the
DPDK change is not needed.

DPDK commit message

net/ixgbe: fix over using multicast table for VF

VMOLR.ROMPE allows a VF to receive packets matching the shared multicast
table.  Leaving it enabled after the VF removes its last multicast
address lets PF or peer-VF table entries continue selecting that VF.

Signed-off-by: Wei Zhao <wei.zhao1@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>

Obtained from:  DPDK (dc5a6e7422)
MFC after:      1 week

ixgbe: fix host interface timeout detection

The host-interface polling loop was scaled from milliseconds to
microseconds, but its terminal test was left using the unscaled timeout.
Completion at that intermediate iteration can be reported as a timeout,
while actual expiry is not recognized and can accept stale status.

Test against the scaled loop bound used by the polling loop.

Fixes:          https://cgit.freebsd.org/src/commit/?id=f46d75c90f5f ("ixgbe: improve MDIO performance by reducing semaphore/IPC delays")
MFC after:      1 week

ixgbe: avoid signed shift when assembling ETrack ID

Obtained from:  Intel ix 3.4.39
MFC after:      1 week

ixgbe: dispatch PBA string reads through EEPROM ops

E610 installs a device-specific PBA string reader, but the public API
always calls the generic implementation.  Dispatch through the EEPROM
operation table so device overrides are honored.

Initialize the generic operation for devices that use the ordinary
EEPROM representation.

Obtained from:  Intel ix 3.4.39
MFC after:      1 week

ixgbe: clear VF head write-back state on reset

VF reset and FLR do not clear the transmit head write-back address
registers.  A previous VF driver can therefore leave DMA write-back
enabled with a stale address for the next driver instance.

After consuming the reset request and disabling the VF queues, clear the
address registers for each queue belonging to that VF.  Derive the queue
count from the active IOV mode so peer queue state is not touched.

Linux commit dbf231af81a7 documents the hardware behavior.  The FreeBSD
implementation follows the local queue mapping and register interfaces.

MFC after:      1 week

ixgbe: Recover legacy VFs from invalid DMA targets

82599 and X540 lack the X550 malicious-driver detector. Detect a VF
whose PCI status reports a received master abort while its transmit
ring has outstanding descriptors and makes no progress across
consecutive samples.

Consume the accepted PCI status latch, gate that VF I/O, and recover
one pending VF per task pass with round-robin selection. This prevents
an unreadable function from starving detection or recovery of other
VFs.

Save the complete writable VF PCI configuration before FLR, restore it
afterward, and verify the hardware-backed Command state. Preserve the
first good snapshot and pending state across reset events until restore
and verification succeed.

Introduce a common I/O-disabled policy bitmask so later quarantine
policy can extend traffic gating without duplicating fault-state
checks.

MFC after:      2 weeks

ixgbe: quarantine repeatedly faulting legacy VFs

A guest can reinitialize after a VF function-level reset and
repeatedly strand an 82599 or X540 PF with invalid descriptor DMA
targets. Count only distinct Received Master Abort events accepted by
the qualified transmit-stall detector and quarantine the VF after five
events.

Preserve quarantine across PF reinitialization, reject reset mailbox
requests, and keep transmit, receive, and clear-to-send disabled.
Recreating SR-IOV clears quarantine. Expose the affected pools through
a read-only bitmap.

After a successful quarantine FLR, leave the function in post-FLR
configuration, explicitly keep decode and bus mastering disabled,
verify the Command register, and refresh its PCI-layer cache so a later
restore cannot re-enable the function.

This addresses CVE-2021-33061 on 82599. Apply the same bounded-failure
policy to X540 as defense in depth; the CVE does not list X540. Intel
documents the 82599 issue in:
http://iommu.com/datasheets/ethernet/controllers-nics/intel/ixgbe/Intel_82599_Application_Note_655276.pdf

MFC after:      2 weeks
Security:       CVE-2021-33061

ixgbe: Re-enable the SFP laser during initialization

ixgbe_if_stop() disables the transmit laser on every 82599 SFP fiber
port, but the iflib initialization path did not re-enable it. Re-enable
the laser before deferred SFP module setup so interface
reinitialization cannot leave either single-speed or multispeed optics
dark.

The hardware wrapper is a no-op when laser control is unavailable. The
placement follows Intel ix-3.4.39; this version deliberately applies to
every SFP port affected by the stop path.

MFC after:      1 week

ixgbe: Defer ECC recovery to iflib

The link interrupt filter performed a full hardware reset in interrupt
context.  This bypassed iflib stop and initialization, including queue
quiescence and restoration of temporary LED state.

Record the ECC event in the administrative request mask and ask iflib
to perform the reset from its taskqueue.  Keep the ECC cause masked
until reset so the intermediate admin pass cannot re-enable a sticky
condition.  Handle ECC independently of Flow Director and in legacy
interrupt mode.

Remove the redundant EICR write; the filter has already cleared the
reported causes.  Also remove the accompanying complement-mask update
of mac.flags.  It set every flag except DOUBLE_RESET_REQUIRED and had
no place in ECC recovery.

MFC after:      2 weeks

ixgbe: Defer firmware recovery transitions to iflib

The firmware-mode callout invoked ixgbe_if_stop() directly.  This
performed a full device reset without the iflib context lock or the
iflib queue lifecycle.  It could also poll the E610 firmware command
interface from callout context while identification was active.

Request an iflib reset from the callout instead.  Reject initialization
while firmware recovery remains active.  This leaves the interface
stopped and lets iflib publish that state.  Request initialization when
firmware exits recovery so an administratively-up interface can recover
without operator intervention.

MFC after:      2 weeks

ixgbe: Defer E610 thermal shutdown to iflib

The E610 firmware event handler invoked ixgbe_if_stop() directly from
IFDI_UPDATE_ADMIN_STATUS().  This reset the device without the iflib
queue lifecycle and left the interface marked running after its hardware
was stopped.

Request an iflib reset instead.  Fail the automatic initialization once
so the reset transaction stops the interface and publishes that state.
A later operator-requested initialization remains possible, matching the
previous recovery policy without bypassing iflib.

MFC after:      2 weeks

Fixes:                https://cgit.freebsd.org/src/commit/?id=fdc1f3450634 ("x86: change signatures of ipi_{bitmap,swi}_handler() to take pointer")
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation

acpi_probe_child() keeps PCI link devices, the RTC,
and docking stations enabled even when _STA reports them not present,
but skipped acpi_parse_resources() for them. With an empty resource
list, resource-based hint matching (BUS_HINT_DEVICE_UNIT) cannot wire
such a device to its hinted unit, and the hinted ISA device is then
created as a duplicate.

Modern AMI firmware reports the PNP0B00 RTC as not present while
handing timekeeping to the ACPI Time-and-Alarm device.

Reviewed by:    adrian, jhb
Differential Revision:  https://reviews.freebsd.org/D58047

Classify I226_LMVP and I226_BLANK_NVM as I226 silicon so they
receive the I226-specific ASPM L1.2 workaround.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=279245
MFC after:      1 week
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2318

When allow-set-mac is disabled, the MAC filter validation condition
rejects the assigned VF unicast address while allowing any different
unicast address. The equality test was accidentally inverted when this
code moved to the boolean address helper.

Accept multicast and the assigned unicast address, and reject other
unicast addresses as intended.

Fixes:          https://cgit.freebsd.org/src/commit/?id=7d4dceec1030 ("ixl(4): Fix VLAN HW filtering")
MFC after:      3 days

The conventional VLAN filter update skipped zero shadow words. Removing
the final VLAN represented by a VFTA word therefore left the hardware
bit programmed even though the software shadow was clear.

Pass the changed word to em_if_vlan_filter_write() and write it even
when its new value is zero. Retained nonzero words continue to be
replayed as before.

I225 devices can incorrectly enter L1 substates while CLKREQ# is
asserted, both while idle and in D3.  Disable ASPM and PCI-PM L1.2 on
I225 to prevent the resulting packet loss.

Keep the I226 workaround ASPM-only because it addresses a separate
traffic exit latency observation.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265714
MFC after:      4 days

igb: preserve coalesced 82576 MDD events

WVBR is read-clear, so reading it from the deferred admin pass loses
earlier queue bits when multiple VF malicious-driver events arrive
before that pass.

Snapshot WVBR in the interrupt filter, translate its staggered queue
bitmap to pool bits, and OR observations into software latches for
deferred notification and recovery.

Retain the one-queue VMDq policy used for mixed-driver safety (the
vswitch cannot handle a 2Q guest loopback to a 1Q guest per errata).

igb: drain stale MDD state before interrupt arm

IOV policy setup can leave MDDET and its read-clear diagnostic
registers populated while the admin vector is masked. Carrying that
state across the unmask can suppress the next spoof-event edge.

Mark initialization for a one-shot drain and consume LVMMC, WVBR when
applicable, and ICR immediately before EIMS/IMS arms the vector.
Preserve the synthetic link-status cause across the arm-time ICR read,
and clear the one-shot latch at reset preparation.

igb: recover retained i350 admin interrupts

I350 can retain EICR.OTHER with MDDET and LVMMC asserted while the
admin vector and legacy cause remain enabled. The anti-spoof filter
continues dropping packets, but no MSI-X is delivered and the spoof
diagnostic is lost.

Preserve the one-shot setup drain across iflib reset preparation,
clear ICR before LVMMC during i350 setup, and kick the enabled admin
vector from each admin pass. The synthetic no-cause interrupt stays
in the filter and also releases a retained MDDET cause.

Keep 82576 drain ordering and stop-time cleanup unchanged.

I225 v1 cannot receive the minimum inter-packet gap required at
2.5 Gb/s.  For affected back-to-back links, Intel recommends using a
15-byte transmit IPG instead of 12 bytes.

Program TIPG.IPGT to 0xb for pre-v2 I225 devices at 2.5 Gb/s and
restore the default at lower speeds.  Avoid penalizing fixed I225 and
I226 parts.

MFC after:      2 weeks

Track RERC separately instead of adding receive errors to the collision
count, and read the previously omitted RXERRC register.  Include RFC in
input errors because CRCERRS does not count bad-CRC runts, implementing
the I225 length-error accounting workaround alongside RUC and ROC.

Stop treating host transmit MAC discards as receive errors.  Expose both
RERC and HTDPMC as dedicated MAC statistics so their overlapping counts
remain available without corrupting aggregate interface counters.

MFC after:      2 weeks

Both these functions have non-static linkage for good reasons, however, their
naming may confuse folk when working with crypto(9) code at global scope.

Borrow the e1000 VLAN filter table

Ambiguous presence of the feature by Intel was settled by DPDK and
emperical testing.

MFC after:      2 weeks
Relnotes:       yes

X550-family malicious-driver detection validates the transmit
context selected by a data descriptor with Check Context set.  ixgbe
sets that bit on every transmit data descriptor, but ordinary PF
packets without a VLAN or checksum offload do not create a context
descriptor.  The empty context then reports an invalid MAC-header
length and blocks the PF queue as soon as MDD is enabled.

Create the existing context descriptor for every PF packet while
SR-IOV is active.  This supplies the required MAC-header length and
keeps MDD from mistaking normal PF traffic for a malicious-driver
event.

MFC after:      1 week

Request the reset through iflib and let the admin task perform the
stop/init under the context lock, matching what the VF and SR-IOV paths
already do.

The assertion is compiled out without INVARIANTS, where the same write
instead resets the MAC and takes the ICH software flag while the queues
stay live and an ioctl or the admin task may be running.

While here also remove unnecessary em_if_init uses:
iflib_if_init_locked() already runs after IFDI_RESUME and
IFDI_MEDIA_CHANGE, so the trailing *_if_init() only added an unstopped
IFDI_INIT that the following iflib_stop() undoes.

MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58628

igc_sysctl_eee() and igc_sysctl_dmac() called igc_if_init() directly.

Request the reset through iflib instead, and skipping while the interface
is down; the new value is picked up by the next init.

Unlike e1000, igc has no ASSERT_CTX_LOCK_HELD and no acquire_swflag
path, so the defect is silent here rather than an assertion failure.

While here also remove unnecessary igc_if_init uses:
iflib_if_init_locked() already runs after IFDI_RESUME and
IFDI_MEDIA_CHANGE, so the trailing *_if_init() only added an unstopped
IFDI_INIT that the following iflib_stop() undoes.

MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58629

gpio: add generic Intel GPIO pin controller framework

Add a platform-independent driver framework for Intel GPIO pin controllers
found on modern Intel SoCs. The driver accesses GPIO pad registers through
ACPI-provided memory-mapped resources and implements the gpio interface [1]
including pin enumeration, capability reporting, configuration, and
read/write/toggle operations. A common data model of communities and pad groups
allows individual SoC-specific drivers to supply their own pad tables and ACPI
hardware IDs while sharing all register-level logic.

[1] https://wiki.freebsd.org/GPIO

Reviewed by:            vexeduxr
MFC after:              1 week
Sponsored by:           Beckhoff Automation GmbH & Co. KG
Pull Request:           https://github.com/freebsd/freebsd-src/pull/2205

gpio: add Intel Alder Lake-N GPIO driver

Add a GPIO driver for the Intel Alder Lake-N platform based on the generic
intelgpio framework. The driver provides pad group definitions for four GPIO
communities covering groups GPP_A through GPP_T, vGPIO and HVCMOS, and matches
ACPI hardware IDs INTC1056, INTC1057 and INTC1085. The kernel module build
infrastructure and the wiring into files.x86 are included.

Reviewed by:            vexeduxr
MFC after:              1 week
Sponsored by:           Beckhoff Automation GmbH & Co. KG
Pull Request:           https://github.com/freebsd/freebsd-src/pull/2205

gpio: add Intel Tiger Lake-H GPIO driver

Add a GPIO driver for the Intel Tiger Lake-H platform based on the generic
intelgpio framework. The driver defines five GPIO communities with pad groups
GPP_A through GPP_K, vGPIO and JTAG, and matches ACPI hardware ID INT34C6.
The kernel module build infrastructure and the wiring into files.x86 are
included.

Reviewed by:            vexeduxr
MFC after:              1 week
Sponsored by:           Beckhoff Automation GmbH & Co. KG
Pull Request:           https://github.com/freebsd/freebsd-src/pull/2205

The VF admin vector carries both link and PF mailbox causes, but the
filter schedules the admin task only for link-status changes. Defer
administration for every interrupt so reset and control notifications
are serviced promptly.

MFC after:      1 week

The shared VF set-RAR helper restores hw.mac.addr when the PF rejects a
requested address, but ixv ignores the error and leaves the interface
link-layer address unchanged. Subsequent initialization repeats the
rejected request while the interface appears to use an address the PF
will not deliver.

Refresh the permanent address returned by the PF after every successful
reset handshake. Copy the resulting PF-approved address back to the
interface and emit the normal link-layer address notification without
re-entering the driver initialization path. This also recovers from a
prior mailbox transport failure or a PF-side reassignment.

Adapt the igb VF address reconciliation added in a6bb3850e7c6.

MFC after:      1 week

The MTA is shared by the PF and all VFs. The VF mailbox handler only
ORs new bits, so hashes survive list removal and VF reset. Conversely,
PF multicast updates replace the whole table with PF-only state and
discard live VF filters.

Rebuild the table from the PF list and every active VF whenever either
changes. Clear VF multicast state during reset and PF reinitialization,
and remove all VF hashes on SR-IOV teardown. Keep the software shadow
and multicast control state synchronized, and avoid writes to unchanged
MTA registers.

Adapt the aggregate desired-state rebuild introduced for igb(4) in
a2ed165f0049 and its write-elision scheme from 350211ab1782 to ixgbe's
shared MTA.

MFC after:      1 week

In preparation of increasing the KVA on powerpc64 to 2TB to mirror
amd64's, rework the 64-bit Book-E pmap to not allocate all page table
pages at boot time, since that would be a waste of a lot of memory.
Instead, allocate all page table pages for the higher levels, leaving
the leaves (page directories) for dynamic allocation.  This cuts the
boot-time page table size down from ~64MB to ~8MB with the current 32GB
KVA size, and bumping to 2TB KVA the boot-time page table is still ~8MB
instead of ballooning to ~4GB of mostly wasted space.

This reflects what amd64 has, and is needed for using GPUs with large
VRAM.

VLAN registration callbacks only update the software shadow, leaving
the PF unaware until a later full initialization.  Initialization then
retries each failed request in a tight loop, while skipping replay
entirely when local hardware filtering is disabled.

Send additions and removals as soon as the desired state changes,
independent of the VF local-filter capability.  Replay the desired
memberships after reset and retry a bounded batch per timer tick.  Stop
after the first failure so a silent PF can consume only one mailbox
timeout per pass, while a responsive PF can drain several requests.
Treat the retry window as a no-progress deadline: advance it when
pending work succeeds so a large backlog can drain, but leave entries
dormant after a sustained failure.

A successful mailbox request wakes a dormant backlog.  Dispatch
timer-driven retries only while iflib marks the VF running, so a stale
timer tick cannot restore PF VLAN state after the stop path resets the
VF.

Because the callbacks now update the PF or retain failed work for
retry, do not restart the VF for VLAN configuration changes.  This
avoids resetting and flapping the interface for every VLAN addition or
removal.

Also keep receive VLAN stripping synchronized in both the enabled and
disabled cases.

Adapt the bounded VLAN reconciliation scheme from igb VF commit
fdce3830d9a6 to the ixgbe VF mailbox.

MFC after:      1 week

pci: Ignore SR-IOV VFs when tuning MPS

The VF Device Control MPS and MRRS fields are reserved and preserved.
VF transactions use the PF MPS, so a hardwired VF value must not be
used to retune the shared PCIe hierarchy.

Document the previously undocumented tuning knob and clarify why a VF
may continue to display its reserved hardwired value.

This fixes an instant crash/reboot on my Zen3 system with 82599 VFs.

MFC after:      1 week

pci: Preserve adjusted PCIe control state

The PCI bus changes live capability registers after the initial
configuration snapshot has been saved. A later driver reprobe restores
that snapshot and can silently undo the adjustment.

Update the cached Device Control and Root Control bits together with
pcie_adjust_config() writes. Route the persistent Maximum Read Request
setter and the bus-owned AER control changes through that helper as
well, so they share the same restore semantics as MPS reconciliation.

Document the persistent-write contract. Merge only explicitly adjusted
bits into the saved image so unrelated or transient bits observed during
the hardware read-modify-write cannot become persistent.

MFC after:      2 weeks

pci: Reconcile MPS before attaching PCIe devices

Reconcile each newly enumerated link as a unit before child drivers
attach. Firmware may leave Bus Master Enable set after handoff, so use
the bus attachment state rather than that bit to identify the cold
phase.

Preserve an established hierarchy during rescan and hot-add. Refuse a
reduction below a switch because recursive enumeration may already have
made a sibling subtree live; lowering only the local port or Root Port
would produce an inconsistent path. Report capability and active-use
conflicts distinctly.

Handle OFW PCI buses that clone the generic enumeration path.

MFC after:      2 weeks

pci: Add a hierarchy-wide MPS limit

Add a boot-time ceiling for MPS reconciliation. Apply it only while an
entire cold-enumerated link can be configured consistently, and leave
an established active path unchanged.

MFC after:      2 weeks

pci: Optionally disable endpoints with unsafe MPS

Keep warn-only behavior as the default. Add an opt-in policy that
clears endpoint decoding and bus mastering when a newly discovered
function cannot match its active path, while never disabling bridge
functions and their subtrees.

MFC after:      2 weeks

pci: Permit function-level reset of 82599 VFs

Intel 82599 supports FLR on VFs but reports FLR support only in the PF
Device Capabilities register.  The VF register therefore leaves the FLR
Capable bit clear, and pcie_flr() rejects the reset.

Intel documents the zeroed VF PCIe capability structure as erratum 35
in the 82599 Specification Update (B0=Yes; NoFix).

Add a positive FLR quirk for the 82599 VF.  Keep the capability check
for every other function, so an unknown nonconforming VF cannot make
pcie_flr() report success when its reset request was ignored.

SR-IOV requires VFs to support FLR, but a clear capability bit cannot
distinguish the 82599's misadvertisement from a VF that fails to
implement it.

MFC after:      1 week

The IOV callback changes the PF pool, virtualization mode, and hardware
queue indices while iflib still considers the old queue layout live.
Teardown likewise leaves the software pool and mode at their SR-IOV
values.

Use iflib stop/mutate/restart transactions for both transitions.
Disable VF DMA and PCI VF Enable before queue reuse, let outstanding
transactions drain, and restore the non-IOV pool and queue indices on
teardown.

Remove the redundant driver-local pci_iov_detach() wrapper; iflib
already performs that check centrally before the driver detach
callback.

It may be possible to avoid some restart in the future on this hardware
pausing DMA and remapping rings but not pursued yet.

MFC after:      2 weeks

A deterministic IOV configuration error currently reaches the driver
only after iflib has stopped the PF. The required cleanup restart then
causes an avoidable carrier flap.

Follow the igb pattern and validate the request in the PCI IOV method
before entering the restart transaction. Reject queue layouts wider
than the selected virtualization pool before they can alias unrelated
82599 registers.

MFC after:      2 weeks

Currently, the FreeBSD driver configures and destroys queues
sequentially by issuing individual Admin Queue (AQ) commands.

During queue teardown (e.g., interface reset), disabling queues
one by one leaves the device in a partially configured state.
Because the device does not yet know that the driver is in the
process of fully unconfiguring all queues, this intermediate
state can trigger transient error logs (such as when queue 0 is
disabled while other queues are still active).

Modify the driver to use Admin Queue batching for both the
creation and destruction of TX and RX queues. Commands are now
queued and kicked together, ensuring the queue configuration changes
are applied atomically and preventing transient errors from being logged.

Signed-off-by: Sujithra Periasamy <sujithra@google.com>

Reviewed by:    markj
MFC after:      1 week
Sponsored by:   Google
Differential Revision:  https://reviews.freebsd.org/D58696

A VF's pci_devinfo references its PF's pcicfg_iov for resource
bookkeeping, but only the PF implements the SR-IOV capability.
pci_cfg_save() and pci_cfg_restore() treated any non-NULL cfg.iov as
an owned capability and accessed the PF capability offset in VF
configuration space. Saving a VF could therefore replace the shared
PF settings with unrelated VF register values.

Skip SR-IOV capability save and restore for PCICFG_VF children. The
generic PCI and PCIe state of the VF remains preserved. This is also
required by drivers that save VF state around a PF-driven
function-level reset.

MFC after:      2 weeks

When link polling loses mailbox clear-to-send or times out, request an
iflib reset instead of continuing with stale VF state.

The driver callback runs after iflib samples reset requests, so requeue
the admin task to make iflib consume the request on its next pass
rather than waiting for an unrelated timer or interrupt.

MFC after:      2 weeks

ixgbe: Apply the 82599 D3 link workaround only for D3

ixgbe_stop_mac_link_on_d3_82599() implements the workaround for
82599 erratum 33.  It forces incompatible auto-negotiation settings
before the device enters D3, and reset clears them when returning to
D0.

ixgbe_if_stop() is also used for ordinary interface reconfiguration
and recovery.  Those paths do not enter D3 and should not program
this power-management workaround.  They continue to stop the adapter
and disable the transmit laser.

Move the call to ixgbe_setup_low_power_mode(), after
ixgbe_if_stop().  This preserves the required ordering for detach,
shutdown, and suspend while avoiding the D3 settings during ordinary
restarts.

MFC after:      2 weeks

ixgbe: Use PF MTU for 82599 VF jumbo policy

The shared maximum frame size is raised by VF LPE requests, so it
cannot describe the PF MTU when enforcing the 82599 PF/VF jumbo
restriction. Consult the PF ifnet MTU instead.

Also correct the API 1.1 and later comparison so a jumbo VF is enabled
when, and only when, the PF itself uses a jumbo MTU. This matches the
policy implemented by DPDK.

MFC after:      2 weeks

ixgbe: Quiesce VFs across PF reset

Stop VF transmit and receive in hardware, clear PF-side mailbox CTS,
and notify active VFs before resetting a PF. A PF reset invalidates VF
queue state, so the no-CTS control message makes cooperative VFs
discard stale state and renegotiate after the PF returns.

The hardware queue gates synchronously prevent further VF DMA. Do not
hold the exclusive iflib context lock for a fixed VF-watchdog interval
after the reset. Report the PF link transition directly instead of
dispatching mailbox work from the stop path, which could otherwise
re-enable VF I/O mid-reset.

The CTS, PF-control, and VF queue controls follow the reset mechanisms
used by DPDK.

MFC after:      2 weeks

netmap: Fix driver name handling

if_initname() requires the caller to ensure that the lifetime of the
interface's name buffer contains that of the ifnet itself.
netmap_vi_create() wasn't respecting that; we were instead passing the
stack-allocated buffer provided by the ioctl handler.

While here, add a check to avoid assuming that the caller-provided
buffer is nul-terminated.

Reported by:    syzkaller
Reviewed by:    vmaffione
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58676

netmap: Fix a race in kqueue registration

We need to acquire the netmap global lock earlier, to avoid racing with
the NETMAP_REQ_REGISTER ioctl handler.

Reported by:    syzkaller
Reviewed by:    vmaffione
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58677

netmap: Handle overflow when computing ring sizes

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297300
Reported by:    Robert Morris
Reported by:    syzkaller
Reviewed by:    vmaffione
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58678

Found with:   clang -Werror=assign-enum

Found with:   clang -Werror=assign-enum

No functional change, as current callers either don't check the return
value or check it against OCS_HW_RTN_SUCCESS only.

Found with:     clang -Werror=assign-enum

DMA channels are allocated by attach_pre but released by queues_free.
When iflib fails after attach_pre and before queue allocation, neither
the old detach nor queues_free path releases them.

Allocate channels with the TX queue state and make queues_free tolerate
partially allocated rings. Use it to unwind allocation failures so TX
rings are also released when RX allocation fails.

An early detach can also precede PHY initialization and interrupt
assignment. Skip absent PHY and channel state, and release the locks
owned by attach_pre on both failure and detach.

MFC after:      2 weeks

Completion queues are allocated by attach_pre but released by
queues_free. An iflib failure between those stages leaks the allocation,
while the original size expression also underallocates the array.

Move completion queue allocation into the TX queue callback, correct its
size, and unwind it with TX state if RX allocation fails. Make interrupt
cleanup tolerate an unavailable array and reuse the array allocated
during device initialization instead of replacing and leaking it.

Release the DMA, multicast, and lock resources owned by a successful
attach_pre during detach. Avoid allocating the statistics DMA area a
second time near the end of attach_pre.

MFC after:      2 weeks

A PF can be resetting, handling a slow link event, or deliberately
withholding mailbox CTS while its VFs enumerate. Keep the VF attached
when the reset handshake is temporarily unavailable so a later if_init
can retry.

Never leave VF hardware running without a negotiated mailbox API: start
hardware only after reset succeeds, stop it when negotiation fails in
attach or init, and defer later recovery through iflib. This prevents a
tight reset loop while preserving recovery when the PF returns.

MFC after:      2 weeks

The iflib conversion records link-status interrupts in the
administrative request mask, but the administrative task did not
consume them.  Timer polling usually hid the omission; frequent mailbox
interrupts could continually rearm that timer and leave cached link
state down after hardware recovered.

Claim request batches atomically, process link-setup dependencies, and
sample hardware before publishing link state.  Bound each invocation to
eight batches and requeue residual work so a continuous producer cannot
monopolize the admin taskqueue.

Queue every link-related request from the legacy interrupt path.
Unlike MSI-X, its threaded continuation services RX and does not enqueue
the admin task.  This restores the event-driven behavior of ix-3.4.39.

Fixes:  https://cgit.freebsd.org/src/commit/?id=b2c1e8e62049 ("ix(4): Run {mod,msf,mbx,fdir,phy}_task in if_update_admin_status")
MFC after:      2 weeks

The aggregate VF mailbox poll includes only VFs whose driver
configuration completed. A configured VF slot whose vf_add callback
failed can nevertheless report reset, request, or acknowledgement
events. Because the mailbox handler skips inactive entries, such an
event remains latched and can retrigger administrative work
indefinitely.

Build the poll masks from every configured VF index and consume reset,
message, and acknowledgement events for inactive entries without
treating them as usable VFs. Use the index rather than the pool because
early vf_add errors precede pool initialization. Also include E610
PFVFLREC in aggregate reset sampling.

MFC after:      2 weeks

Register the vmm module handler after both the bundled device drivers
and SMP. On platforms without EARLY_AP_STARTUP, SI_SUB_SMP follows
SI_SUB_DRIVERS; using the later subsystem preserves the
smp_rendezvous() requirement.

The resulting reverse unload order performs IOMMU cleanup while every
IVHD softc remains valid. Refuse an independent IVHD detach while
translation state remains initialized.

MFC after:      2 weeks

Do not instantiate an interrupt-remapping context for a unit whose IRTE
support is disabled. In that mode the caller must retain the ordinary
interrupt path.

Reviewed by:    kib
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58725

The iflib Flow Director path does not assign filters using the
absolute queue and pool identifiers required by SR-IOV.  Reject the
combination during preflight validation rather than allowing an
unsupported configuration to alter the PF receive path.

The loader tunable is fixed before VFs can be created, so validation
also prevents the reverse ordering of this combination.

MFC after:      2 weeks

The flow_control and hdr_split variables have never been read.  VF
flow control is controlled by the PF, while implementing header split
would require receive-path support that ixv does not provide.

MFC after:      2 weeks

The shared ixgbe transmit path already creates SCTP context
descriptors, and the hardware exposes the same checksum capability to
VFs.  Advertise it through iflib as the PF driver does.

MFC after:      2 weeks

TXDCTL programming is family dependent.  82543 erratum 35 and
82544 erratum 20 require WTHRESH to remain zero; a nonzero value
can corrupt descriptor writebacks and hang the controller.  Leave all
descriptor-control thresholds at their reset values on 82542, 82543,
and 82544.

On the remaining em controllers, retain the established PTHRESH=31,
HTHRESH=1, WTHRESH=1, and descriptor granularity policy.  Several
legacy specification updates identify full descriptor writeback as a
workaround for transmit descriptor-queue errata.

TXDCTL bit 22 is also family dependent.  It is COUNT_DESC on the
82571 family and 80003ES2LAN.  Intel shared initialization explicitly
sets raw bit 22 on both transmit queues of every supported ICH/PCH
generation, although the integrated public documentation marks it
reserved.  Preserve that required setting when iflib programs the
thresholds, as DPDK does.  Clearing it caused a persistent I219
transmit stall under descriptor pressure.

The combined em/igb setup also wrote LWTHRESH=1 on every em
controller.  The driver does not enable the TXD_LOW interrupt
controlled by that field.  Enumerate every supported em MAC type and
leave the unused low-water threshold disabled.

This keeps the legacy descriptor-writeback safety policies separate
from igb sparse-RS operation while programming only the fields
appropriate to each family.

MFC after:      2 weeks

Jumbo receive tuning on integrated controllers enabled PTHRESH without
a nonzero HTHRESH, contrary to the hardware programming requirements.
It also covered only the integrated MAC generations present when the
workaround was added.  Enumerate every jumbo-capable ICH and PCH type
and program PTHRESH=3 with HTHRESH=1.  Linux fixed the same HTHRESH
omission in b701cacdbcfb.

The 82574 path combined threshold values with the reset values using
bitwise OR.  Requesting WTHRESH=4 while the reset value was one thus
programmed five.  Clear the complete threshold fields before installing
the established PTHRESH=32, HTHRESH=4, WTHRESH=4 descriptor-granularity
policy.

MFC after:      2 weeks

iflib requests transmit completion status only on selected descriptors.
Program a zero writeback threshold so igb hardware honors those sparse
RS bits instead of writing back every descriptor in threshold-sized
batches.

Use the existing family specific prefetch threshold: eight descriptors
on most controllers and 20 on I354, with a host threshold of one.  These
values match the Intel-derived Linux and DPDK drivers.  Their nonzero
writeback settings are not appropriate here because those drivers set
RS on every packet.

A zero writeback threshold also avoids depending on interrupt timer
flushes affected by 82576 specification update erratum 26.  Remove the
old IGB_TX_WTHRESH macro as well.  It has had no callers since the iflib
conversion, so its 82575 conditional no longer implements any policy.

MFC after:      2 weeks

82576 specification-update erratum 26 says MSI-X EITR expiration can
fail to trigger receive descriptor writeback.  A WTHRESH above one can
therefore leave received packets invisible until the threshold fills.

The shared threshold macros selected policy by enum ordering, so an
82576 VF fell into the generic WTHRESH=4 case.  VFs always use MSI-X
and require the same WTHRESH=1 workaround as the PF.

Use PTHRESH=8 for 82575 and 82576 PFs and VFs, matching DPDK and the
current Linux PF driver.  The legacy FreeBSD PF and Linux igbvf value
of 16 thrashes limited descriptor cache; no specification or erratum
requires it.  Retain the i354 PTHRESH=12 exception.

Enumerate every supported igb PF and VF MAC type so each receives its
intended policy.  Also clear every threshold bit before installing the
new values.  The old mask retained the high WTHRESH bit, and 82575
uses six-bit fields while later controllers use five-bit fields.

MFC after:      2 weeks

The transmit-ring setup was copied from the e1000 path.  On I225
and I226, bits 22 through 24 are reserved and bit 25 enables the
queue; it is not a legacy low-water threshold.  Correct the field
masks, remove the nonapplicable legacy definitions, and program only
defined fields.

Use PTHRESH=8 and HTHRESH=1.  Keep WTHRESH at zero so the hardware
honors sparse RS descriptors issued by iflib.  Linux and DPDK use a
writeback threshold of 16, but request status on every packet.  A
nonzero threshold makes hardware ignore individual RS bits and is
unsuitable for the iflib completion model.

The receive-ring setup likewise used a magic mask that left bit 20
of the five-bit WTHRESH field untouched.  Define the receive threshold
fields and replace them exactly before installing the established
PTHRESH=8, HTHRESH=8, WTHRESH=4 policy.

MFC after:      2 weeks

PTHRESH controls when the device prefetches transmit descriptors,
HTHRESH controls how many host descriptors must be ready, and WTHRESH
controls completion writeback batching.

iflib places RS on selected descriptors and reclaims through those
checkpoints.  The data sheets require WTHRESH to be zero when software
uses RS.  Clear WTHRESH while retaining the established PTHRESH 32 and
HTHRESH 1 fetch policy.

This also follows DPDK in pairing sparse RS descriptors with
WTHRESH zero.  DPDK defaults to 32/0/0, while Linux ixgbevf uses
32/1/8.  The 32/1/0 setting preserves FreeBSD's prefetch policy and the
data-sheet requirement that HTHRESH be nonzero when PTHRESH is used.

MFC after:      2 weeks

ixv uses one queue set on 82599 and X540 VFs and assumes two on
X550-family VFs.  The PF reports the queues assigned to each VF with
GET_QUEUES after mailbox API 1.1 negotiation.

Query the PF during attach.  Bound symmetric iflib queue sets by the PF
grant and available MSI-X data vectors.  Retain one queue set per data
vector: ixgbe VFs expose at most three vectors and one is reserved for
the mailbox.  The hardware permits each pool to use a subset of its RSS
queues, so a two-queue ceiling is valid when the PF assigns four.

This enables the second data vector on 82599 and X540 while avoiding an
assumed second queue when an X550-family VF is granted only one.  Keep
the existing family limits if the mailbox is unavailable or the PF uses
an older API.

MFC after:      2 weeks

iflib clears IFF_DRV_RUNNING before the driver stop callback but leaves
IFF_DRV_OACTIVE set.  Consequently, an already queued admin task can
run after the VF reset.  If that task consumes a pending timer sample,
it can retry failed VLAN mailbox operations and restore PF filters for
the stopped VF.

Continue sampling statistics, but only run the VLAN retry worker while
the interface is running.

Expose the cached per-VF configuration through the iflib VF status
method.  Report mailbox handshake state, MAC address, access or trunk
VLAN mode, hardware queue count, administrator policy, and MDD blocking
state without issuing mailbox requests or reading hardware registers.

ixgbe: Report SR-IOV VF status

Expose cached VF configuration, policy, and runtime state through the
iflib VF status method.  Include access or trunk VLAN mode, the queue
count selected by the current virtualization mode, negotiated mailbox
API, whether traffic is enabled, and the MDD-blocked and quarantine
state.

The query runs under the iflib context lock and does not issue mailbox
requests or read hardware registers.

ixgbe: Add missing mailbox API 1.6 definition

The SR-IOV status change reports mailbox API 1.6 but omitted its enum
definition, leaving main unable to compile.

API 1.6 is an established ixgbe mailbox wire revision.  Add it at the
end of the revision enum, before the unknown sentinel as required by
the stable numbering contract.  Naming the revision does not enable
negotiation or operations which will come with the E610 support.

Reported by:    Herbert J. Skuhra <herbert@gojira.at>
Fixes:  https://cgit.freebsd.org/src/commit/?id=c30021fe0df9 ("ixgbe: Report SR-IOV VF status")

video: add generic video(4) capture framework

Add a new video(4) framework that provides /dev/videoN, buffer
management, mmap lifetime, and V4L2 ioctl dispatch for video capture
drivers. Hardware drivers implement struct video_hw_ops callbacks
and use video_buf_acquire/write/done to deliver frames.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58367

video: disable the static assertions for now

The previous version of this work included the definitions but
not the static asserts.  It's tripping up in some CI builds, likely
due to compat API building.

Since this isn't any more or less broken than before, disable the
static assertions until we figure out a proper path for this.

Fixes: https://cgit.freebsd.org/src/commit/?id=9c9428825f4c55e3cb37412c661bb9d385db4c68 (video: add generic video(4) capture framework)

Replaced the monolithic cdevsw implementation with the video(4)
framework.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58368

Replaced the monolithic cdevsw implementation with the video(4)
framework.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58369

Raising UVIDEO_NFRAMES_MAX from 40 to 128 in 3b6f833c95eb improved
throughput on xhci but made every camera on an ehci bus fail to
stream. Integrated webcams became unusable.

Measured on a MacBookPro9,2 with two ehci(4) FaceTime HD cameras and an
xhci(4) Logitech C920:

                          128            32
  ehci, 12 captures       0 ok           12 ok
  xhci 1920x1080          5 fps           5 fps
  xhci 1280x720          10 fps          10 fps

Fixes:          https://cgit.freebsd.org/src/commit/?id=3b6f833c95eb

Reviewed by:    bapt
Differential Revision:  https://reviews.freebsd.org/D58501

Disabled the IR DMA channel on the error path, which clears the flag and
frees the descriptor blocks before the chunks go away.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58502

Reviewed by:  adrian, bapt
Differential Revision:  https://reviews.freebsd.org/D58503

Fix two small bugs affecting the event parsing of AMD L3 counters.
AMD's manual and JSON disagree about the naming scheme on recent
processors.  I use the naming scheme present in the recent PPRs to be
consistent, so in the JSON parser we rename 'allslices' to 'allsources'
just as we already do with sliceid and sourceid.  Also ensure that we
parse the 0x prefix present in the newer JSON files.

Reviewed by:    mhorne
Sponsored by:   Netflix
MFC after:      1 week
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2180

A link flap left nothing in the log to work from.  Both the link up and
link down messages were gated on bootverbose while the message for a
speed change that keeps carrier was not, so a default kernel was silent
about a flap yet loud about a downshift -- the inverse of what an
operator wants.  The generic message from if_link_state_change() carries
no speed, so gating the driver's own left the negotiated rate
unrecorded.  Report both transitions unconditionally.

Say more than the rate.  aq_hw_get_link_state() already negotiates flow
control and throws it away, and Atlantic 2 reports duplex and EEE in the
same link status word the rate comes from; decode them through a new
get_link_info firmware op and name all of it on the up transition.  EEE
matters for a flap: low power idle transitions are a common source of
marginal link trouble on multi-gigabit copper, and whether it was active
is otherwise invisible.

Give the down transition a cause.  The PHY global fault code was only
consulted from the thermal state machine, so an ordinary link loss
reported nothing at all.  Read the fault code, the firmware link state
and the PHY temperature once per transition and append whatever is
available.  The firmware raises a fault one poll after it drops the
link, so a thermal trip usually shows only its temperature here and
aq_thermal_poll() names it on the following poll; the temperature alone
is enough to separate a hot PHY from a cable event.

Warn before the PHY trips rather than only after.  The Atlantic 2 health
monitor word carries a hot warning bit next to the ready and fault bits
that nothing decoded.  Report both edges of it from the thermal poll, so
an adapter that is approaching its shutdown threshold says so while the
link is still up.

Expose the firmware's own link transition counters.  The Atlantic 2 A0
statistics layout opens with link_up and link_down, which were read out
of the firmware on every statistics poll and discarded.  Publish them as
dev.aq.N.fw_link_up and fw_link_down so a single flap can be told from a
link that has been flapping all night.  The B0 layout has no equivalent,
so the op reports ENOTSUP there and the nodes are not created, matching
how the temperature node is handled.

Stop announcing a link state that was never read.  The return value of
aq_hw_get_link_state() was discarded, so a failed read would have been
announced as link down.  No firmware backend can fail that call today --
all three decode a register with no error path -- but the caller no
longer depends on that, and it says so once if it ever starts failing.

Report the hardware failures that were being discarded.  The driver
already reports the errors it keeps, so what stayed quiet was the set of
calls whose result was never examined at all.  None of these are
expected to fail, which is precisely why a failure needs to say so: each
one leaves the interface running but misconfigured in a way that
presents as a network problem rather than a driver problem.  aq_if_init()
discarded aq_hw_start(), aq_hw_rss_hash_set(), aq_hw_rss_set() and
aq_hw_udp_rss_enable(), so a datapath that never started or an
indirection table that was never programmed showed up only as an
interface that passes no traffic or delivers every flow to one queue.
aq_mc_filter_apply() discarded aq_hw_mac_addr_set(), so a multicast
address the stack believes is programmed could silently not be; report
the address that failed and leave the filter slot for the next one
instead of burning it.  aq_update_vlan_filters() reported only the last
of its three register writes.  aq_if_stop() discarded both ring stop
calls and the MAC reset, and a MAC that did not reset can still be
mastering the bus.  aq_if_detach() and aq_if_suspend() discarded
aq_hw_deinit().  The interrupt moderation update on a link speed change
was dropped as well; it runs only on a transition, so reporting it
cannot become noisy.

aq_if_attach_pre() discarded aq_hw_capabilities(), which is the only
behavioral change here: it now fails the attach rather than continuing
with an unset media type and an empty link speed mask, which would
attach an interface that can never negotiate a link.  It returns an
error only for a device the probe table does not cover, so it is not
reachable in practice.

Document the resulting sysctls, along with the existing temperature and
tracing nodes, which had no manual page coverage.

Tested on an AQC113C (Atlantic 2 B0, firmware 1.5.38).  Link up reports
"speed=10000, full-duplex, flowcontrol none, EEE off", and "speed=1000"
after a forced renegotiation, so the rate and duplex are read rather
than assumed.  A cable pull reports "link DOWN, F/W link state 0, temp
59 C" with the PHY fault clause correctly absent, which is what
separates a cable event from a thermal trip.  The B0 interface reports
ENOTSUP for the link counters, so those two nodes are correctly not
created.  Traffic is unaffected: ten flows spread over all eight RX
queues with no errors and no drops.

Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D58749

Signed-off-by: Nick Price <nprice@FreeBSD.org>

A VF reset can sanitize its retained queue registers even when the PF
does not complete the cooperative mailbox handshake.  Keep those two
states separate.  Do not program or enable the rings until both queue
sanitation and mailbox initialization have succeeded.

Report either initialization failure to iflib so the interface remains
stopped.  Stopped admin and media-status passes now publish cached
link-down state without polling the mailbox.  While the VF remains
administratively up, retry complete initialization after 250 ms, one
second, four seconds, and then at a capped eight-second interval.
Conditional iflib reset requests ensure an intervening administrative
down cancels a queued retry.

Avoid a redundant mailbox reset in the stop half of an immediate iflib
reinitialization; the following init performs the required reset.
Preserve the reset on an ordinary administrative stop and keep the
existing bounded queue-sanitation retry policy independent from
mailbox liveness recovery.

A failed VF reset or mailbox API negotiation currently returns from
the void ifdi_init callback. Iflib then marks the interface running
even though ixv left its adapter stopped. Stopped media queries can
continue polling the PF, and no timer remains active to retry when the
PF returns.

Track mailbox readiness and report unsuccessful initialization to
iflib. Stopped admin and media-status passes now publish cached
link-down state without touching the mailbox. While the VF remains
administratively up, retry complete initialization after 250 ms, one
second, four seconds, and then at a capped eight-second interval.

Preserve the requested MAC across reset, then program it once after
mailbox API negotiation. The previous two pre-reset requests each
could wait a full mailbox timeout after an established PF disappeared,
holding the iflib context lock for about two seconds before the reset
handshake.

Avoid a redundant VF reset in the stop half of an immediate iflib
reinitialization. Also remove the stop-time RAR mailbox request: reset
has already discarded CTS at that point, and successful initialization
restores the current address. Retain a reset for an ordinary
administrative stop when the mailbox was established.

MFC after:      2 weeks

PSRTYPE is indexed by pool in VMDq+RSS mode, and its RQPL
field selects the number of receive queues available within the pool.
The PF occupies the last pool, but the driver programmed pool zero and
left the PF RQPL value at zero.  As a result, all PF receive traffic
was directed to its first queue while SR-IOV was enabled.

Program PSRTYPE for the PF pool and encode its allocated receive queue
count.

MFC after:      2 weeks

HWRM failures currently return from the void ifdi_init callback.
iflib then marks the interface running and enables interrupts despite an
incomplete ring or VNIC setup.

Move the hardware setup into an error-returning helper.  The ifdi_init
wrapper can report failure through iflib_init_failed(), while firmware
recovery can propagate the same error through bnxt_open().  Also clear
the initialized state after partial setup is torn down.

MFC after:      2 weeks

The primary and mirror-VSI ifdi_init callbacks can return early when
reset state or hardware queue and filter setup prevents initialization.
Iflib then marks the interface running and enables interrupts although
the driver did not finish bringing it up.

Report each non-detach failure through iflib_init_failed().  Keep the
existing ice reset and subinterface-reinitialization machinery
responsible for scheduling recovery.

MFC after:      2 weeks

The 82542-specific setup routine unconditionally reads the NVM
default, overwriting a flow-control mode selected by software.  It
also removes transmit PAUSE support from all 82542 revisions even
though the hardware restriction applies only to rev 2.0.

Resolve the NVM default only when requested, scope the transmit
restriction to rev 2.0, and replace integer bit masking of the enum
with explicit valid mode transitions.  This restores the behavior
from before the Intel shared-code split and resolves -Wassign-enum.

Reported by:    glebius
MFC after:      2 weeks

ufshci: fix data direction encoding for read commands

The data_direction field in the UTP Transfer Request Descriptor is only
2 bits wide ([26:25]). UFSHCI_DATA_DIRECTION_FROM_TGT_TO_SYS was defined
as 0x10, which truncates to 0b00 (No data transfer) when stored into the
2-bit field, so every read command was described to the controller as
having no data phase. Only writes (0b01) happened to be encoded
correctly.

Define all values as 2-bit binary literals, matching the existing
RESERVED = 0b11 entry, so read is encoded as 0b10 as required by the
specification.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58652

ufshci: abort submission when payload DMA mapping fails

When bus_dmamap_load_mem() failed, ufshci_req_queue_prepare_prdt()
manually completed and released the tracker, but its caller kept going:
it built the UTRD, set the slot back to SCHEDULED, and rang the
doorbell for a tracker whose request had already been freed. Return the
mapping error and stop the submission so the released tracker is not
resurrected.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58653

ufshci: fail attribute reads on a non-zero config result code

ufshci_uic_send_cmd() only logged the error code and returned success,
so a failed DME_GET gave its caller a stale value as if it were valid.
The gear and lane settings could then be programmed from that garbage.

Return ENXIO for reads instead. Writes keep logging and continuing,
because a device may reject an optional attribute and that must not
fail bring-up.

Sponsored by:           Samsung Electronics
Reviewed by             imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58654

ufshci: handle controller command submit failures

Return submission errors from the controller command helpers and
propagate them to polled callers before waiting for completion. Free
requests that never enter a hardware queue so failure paths do not leak
or panic after the poll timeout.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58655

ufshci: fix SCSI I/O request failure cleanup

ufshchi_sim_scsiio() did not check the M_NOWAIT request allocation
for NULL. The CDB validation and submit failure paths also returned
without freeing the request.

Fail the CCB when the allocation returns NULL. Free the request on
every failure path. Mark the CCB as queued right before the submit,
so the failure paths above do not need to touch that flag.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58656

ufshci: free the lookup path when the periph search times out

ufshci_sim_find_periph() freed the lookup path only when it found the
periph. The timeout path returned without freeing it and leaked the
path. Free the path at the single exit instead.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58657

ufshci: fix WLUN periph reference counting

The driver stored the WLUN periph pointer without holding a reference,
so the pointer went stale when the pass(4) device went away. In
addition, ufshci_sim_send_ssu() released a reference that it had never
acquired.

Define a simple ownership rule. ufshci_sim_find_periph() acquires the
periph and returns it. The cache owns one reference. The controller
destructor drops it with cam_periph_release() before taking the SIM
lock, since the release takes the CAM device lock by itself.
ufshci_sim_send_ssu() acquires its own reference and releases it when
done. Reuse the cached periph instead of searching again, so the old
reference is not leaked.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58658

ufshci: free the correct address when DMA load fails

The bus_dmamap_load() error paths passed hwq->utrd and req_queue->ucd
to bus_dmamem_free(), but both pointers are only assigned after a
successful load and are still NULL at that point. The freshly
allocated memory was leaked. Free the local buffer instead.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58659

ufshci: tolerate partially constructed queues in SDB teardown

When attach fails, ufshci_req_sdb_destroy() runs on a partially
constructed queue, and it runs twice: once from the construct error
path and once from the controller destructor.

Make that safe: NULL-check each resource before freeing it and clear
the pointer afterwards, so a second call finds nothing to do. The
construct error label no longer frees the command descriptors itself,
which fixes a double free of ucd_bus_addr. Also destroy the payload
DMA tag, which was previously leaked. Drop the mtx_initialized()
checks: the locks are always set up before any failure path can reach
the destroy.

Attach can also fail before the queues were constructed at all. The
destructor would then call a NULL qops.destroy pointer, so skip the
destroy when the queue was never set up.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58660

ufshci: check SDB queue allocations for failure

The hardware queue and ucd_bus_addr allocations use M_NOWAIT but were
used without a NULL check, and the payload bus_dmamap_create() return
value was ignored, so a failed allocation was only discovered by
faulting on it later. Fail the construction instead. The teardown
path handles the partially constructed queue.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58661

ufshci: do not free the devq twice on SIM attach failure

cam_sim_free() with free_devq set already frees the devq, so the
following cam_simq_free() call on the xpt_bus_register() and
xpt_create_path() failure paths was a double free. Also clear
ctrlr->ufshci_sim so a later ufshci_sim_detach() does not operate on
the freed SIM.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58662

ufshci: initialize alloc_units before the dedicated-buffer scan

If every unit descriptor read failed in the LU-dedicated WriteBooster
scan, alloc_units was used uninitialized. Start it at zero so that case
is treated as a zero-sized buffer and WriteBooster is disabled.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58663

ufshci: byte-swap big-endian UPIU fields

The UPIU wire fields are big-endian. The task management and query
builders wrote host-order values into them. The completion paths also
read the results back without conversion. On a little-endian host an
ABORT_TASK carried a swapped task tag and LUN, a query carried a
swapped length, and attribute reads returned swapped values. Tolerant
devices masked most of the damage.

Convert with htobe*/be*toh at the wire boundary, as ufshci_sim.c
already does for its fields.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58664

ufshci: initialize desc_size for non-descriptor query requests

The flag and attribute query builders left param.desc_size
uninitialized, so stack garbage was sent as the query UPIU length
field. Devices generally ignore the length for these opcodes, which
hid the bug. Zero it explicitly.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58665

ufshci: read UIC command results while holding the lock

The UIC result registers (UICCMDARG2/3) are only valid between a
command's completion and the next command's submission. They were read
after uic_cmd_lock was dropped, so a concurrent UIC submitter could
overwrite them in between. Read them into locals before releasing the
lock.

Also mask the generic error code to its [7:0] field when checking it,
so unrelated bits in UICCMDARG2 (such as the attribute set type echoed
for DME_SET) cannot be mistaken for an error.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58667

ufshci: check completions under the queue lock

The completion scan held only the recovery lock. The submit path sets
a slot to SCHEDULED and then rings the doorbell, both under the queue
lock. A scan running between those two steps saw a SCHEDULED slot with
a clear doorbell and completed a command the device had not started.
The command failed with OCS 0xf, and a reused slot could return wrong
read data.

Check the slot state and the doorbell under the queue lock. The submit
path holds it across both steps, so a half-submitted slot can no
longer be seen. Found with fio randrw verify on QEMU.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58668

ufshci: release the CCB after sending a start stop unit command

ufshci_sim_send_ssu() got a CCB from cam_periph_getccb() but never
returned it. Each call leaked the CCB and one slot of the device's
CCB allocation budget. When the budget runs out, the next
cam_periph_getccb() waits forever and the suspend path hangs.

Release the CCB while the periph lock is still held, as the other CAM
periph drivers do.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58669

ufshci: free the taskqueue on detach

ufshci_ctrlr_destruct() never freed the taskqueue. Every load and
unload cycle leaked the taskqueue and its kernel thread. A task that
was still queued could also run after the module was gone.

Free the taskqueue in destruct. Do it after the interrupt teardown
so nothing enqueues new work. A reset task that is still queued at
this point races the queue teardown. That race is older than this
change. The planned in-flight recovery rework will close it.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58670

ufshci: do not reset the device in the XPT_RESET_DEV handler

CAM calls the SIM action callback with the SIM lock and the CAM
device lock held. The XPT_RESET_DEV handler called
ufshci_dev_reset(), which sleeps on device commands. Sleeping there
panics when another thread contends for the lock: "panic: sleeping
thread holds CAM device lock".

Report success without touching the device, as nvme_sim(4) does.
A real device reset needs the controller reset path. That rework is
planned together with in-flight request recovery.

Sponsored by:           Samsung Electronics
Reviewed by:            imp (mentor)
Differential Revision:  https://reviews.freebsd.org/D58671

A PF reset or loss of virtchnl service can make visible interface
initialization wait up to ten seconds and then return from the void
ifdi_init callback.  Iflib consequently marks the interface running even
though its queues were not initialized, and no retry is scheduled when
the PF returns.

Check reset readiness without polling during reinitialization, propagate
queue-message submission errors, and bound a silent enable or disable to
one mailbox timeout.  Report unsuccessful initialization to iflib and
publish link-down state without polling the stopped mailbox.

A VFLR also discards the Admin Queue and permits the PF to replace the
VF VSI.  Track when full virtchnl rediscovery is required, renegotiate the
API version, refresh and validate the VF resources before using a cached
VSI ID, and replay the MAC and VLAN filters cleared by reset.  Bound each
runtime discovery attempt while preserving the existing attach-time wait.

While the VF remains administratively up, retry complete
initialization after 250 ms, one second, four seconds, and then at a
capped eight-second interval.

MFC after:      2 weeks

ixl uses head writeback by default.  Hardware publishes the transmit
ring head through DMA only after completing a descriptor marked RS.
Marking every packet requested much more frequent head updates than
iflib needs to reclaim descriptors.

iflib marks selected packets with IPI_TX_INTR as completion
checkpoints.  It forces a checkpoint as deferred work or ring pressure
grows.  Retain EOP on every packet, but set RS only at those
checkpoints.  This batches head writebacks while preserving bounded
descriptor reclamation.

The optional descriptor writeback mode benefits as well.  ixl already
recorded only IPI_TX_INTR descriptors in its report-status queue, so
status written for every other packet was not inspected.

DPDK uses the same sparse RS design.  Let iflib choose the adaptive
interval for FreeBSD.  This is a PCIe/memory bandwidth savings.

MFC after:      2 weeks

iavf uses descriptor writeback by default.  Hardware writes completion
status into a transmit descriptor only when it completes a descriptor
marked RS.  iavf marked every packet RS even though its report-status
queue recorded and inspected only descriptors selected by iflib.  The
other completion writes could not help reclaim descriptors.

iflib marks selected packets with IPI_TX_INTR as completion
checkpoints.  It forces a checkpoint as deferred work or ring pressure
grows.  Retain EOP on every packet, but set RS only at those
checkpoints.

The deprecated head-writeback option on 700-series VFs gets the same
batching: each RS checkpoint permits hardware to publish the completed
ring head.

DPDK uses the same sparse RS design.  Let iflib choose the adaptive
interval for FreeBSD.  This is a PCIe/memory bandwidth savings.

MFC after:      2 weeks

The VF array is zeroed at allocation, but its sysctl contexts were
only populated after each VF was successfully added.  If VF setup
failed, IOV teardown still passed every requested VF context to
sysctl_ctx_free().  An untouched context is not an initialized empty
TAILQ and caused a page fault during teardown.

Initialize every VF context with the array so both successful setup
and partial-failure cleanup have a valid lifetime.

MFC after:      2 weeks

pci_iov_enumerate_vfs() logged a failed VF creation or driver
configuration but still reported the whole SR-IOV configuration as
successful.  The PF remained enabled with the requested NumVFs and
driver state even though one or more VF children were absent.

Make VF enumeration atomic.  Delete children created by the failed
attempt, invoke the PF driver cleanup, disable VF memory space and VF
Enable, release the IOV resources, and return the original error to
iovctl.  Also treat failure to create a VF child as an error instead
of silently accepting a partial configuration.

MFC after:      2 weeks

Bound variable-length virtchnl messages before computing their expected
length, following the newer Intel virtchnl implementation.

Validate VF ring sizes and alignments before programming HMC contexts.
DPDK uses 128-byte ring alignment and 64 through 8160 descriptors;
the virtchnl ABI further specifies TX multiples of 8 and RX multiples
of 32.  Preserve the 4096-descriptor limit on X722.

Validate queue bitmaps before changing any rings, validate all queue
and interrupt contexts before applying a request, and reject invalid
RSS table entries.  Also avoid sending an ACK after VLAN-strip setup
fails and reply to delete-VLAN errors with the correct opcode.

These checks prevent malformed or oversized requests from an untrusted
VF from partially programming resources outside its allocation.

MFC after:      2 weeks

ixl: Make VF reset resource reconstruction fallible

Treat each stage of VF reset and VSI reconstruction as fallible.  Keep
the VF out of VFACTIVE when PCIe drain, reset completion, VSI
release, or VSI allocation fails, following the DPDK PF reset model.
Propagate initial reset failures back through pci_iov_vf_add and unwind
the VF queue allocation.

Free the old software filter list before initializing a replacement VSI.
ixl_init_filters() previously replaced the list head without freeing its
entries, so every VF FLR leaked all MAC and VLAN filter objects.  Reset
the associated counters and VLAN bitmap with the list.

Avoid allocating an initial VSI only to destroy it during the required
initial VF reset, and remove redundant broadcast/filter programming from
VSI setup.  Also delete a partially created VSI when later Admin Queue
setup fails.

MFC after:      2 weeks

ixl: Enforce VF VLAN policy

Add access and trunk VLAN policy to the SR-IOV schema.  Access VFs
use a hardware PVID and cannot alter their VLAN membership.  Trunk VFs
may register up to 16 VLANs, while VLAN 0 remains implicitly admitted
for untagged and priority-tagged traffic.

Enable hardware VLAN anti-spoofing and maintain the MAC-by-VLAN filter
cross-product used by DPDK.  Apply Linux's untrusted-VF limits of 18 MAC
addresses and 16 VLANs so one guest cannot consume the shared PF filter
table without bound.

Report the effective policy through the VF status interface and document
the iovctl schema.

MFC after:      2 weeks
Relnotes:       yes

ixl: Rebuild VF resources after a PF reset

A PF or EMP reset destroys the firmware switch topology, including
every VF VSI.  The driver rebuilt only its PF VSI and left configured
VFs with stale switch element and VSI identifiers.

Notify VFs before a driver initiated reset, recreate the IOV VEB, and
rebuild each configured VF VSI and queue mapping after the PF switch
is restored.  Keep a VF out of VFACTIVE if its reconstruction fails
so one failure cannot expose incomplete resources or prevent the PF
and other VFs from recovering.

Invalidate cached VF firmware identifiers and runtime state before
recreating the VEB.  If VEB creation itself fails, teardown and mailbox
paths can no longer use pre-reset SEIDs or VSI data.

Factor the common VEB setup out of IOV initialization so initial
setup and post-reset reconstruction use the same topology and filter
sequence.

MFC after:      2 weeks

ixl: Report PF initialization failures to iflib

ixl_if_init() returned early after AdminQ reconstruction, LAA, or
VSI initialization failures.  Since IFDI_INIT has no return value,
iflib then marked the interface RUNNING and enabled its interrupts
and timers despite the incomplete hardware state.

Use iflib_init_failed() on each incomplete path.  Also stop at the
first ring-enable error and tear down any partially enabled rings
before reporting failure.  This keeps the interface stopped and
makes a later initialization attempt start from a bounded state.

MFC after:      2 weeks

ixl: Track and recover MDD-blocked VFs

The hardware identifies each VF with TX and RX malicious-driver
status latches, but the driver combined all events into one counter
and reported only the last VF found.  It also did not record that
hardware had blocked the VF, leaving the condition invisible to
management tools.

Consume every PF and VF latch, keep per-direction VF counters,
rate-limit per-VF diagnostics, and report the blocked and
traffic-enabled state via the VF status interface.  Clear the
software block only after a successful VF or PF reset reconstructs
its resources.

Match Linux i40e policy by leaving a detected VF blocked by default.
Add an opt-in hw.ixl.mdd_auto_reset_vf tunable that notifies and
resets the VF for installations that prefer availability.  DPDK
provides the register clear and per-VF attribution precedent; Linux
provides the recovery policy.

MFC after:      2 weeks

ixl: Quiesce VF DMA before a PF reset

A PF reset has a warning interval before the hardware reset begins.
Cooperative VF drivers respond to the reset event by stopping and
releasing their receive buffers, but notifying VFs did not stop the
hardware queues.  An active VF could therefore DMA through its old
rings into freed mbuf clusters during the warning interval.

Put every enabled VF in reset, drain its PCIe transactions, disable its
queues, wait for receive queue shutdown, and drain transactions again
before tearing down the PF HMC and AdminQ.

Hold VFs in reset again while rebuilding the firmware topology.  Release
VF reset before programming the replacement VSI and queue mappings,
since VF reset clears those registers, and publish VFACTIVE only after
reconstruction succeeds.  Leave a VF held in reset if rebuilding it
fails.

Fixes:          https://cgit.freebsd.org/src/commit/?id=983e628a0c47 ("ixl: Rebuild VF resources after a PF reset")
MFC after:      2 weeks

A PF link event remains cached while a VF is administratively down.
Media status queries called iavf_update_link_status() and published
that cached state as link-up, while the stopped admin path immediately
published link-down.  Consumers reacting to link events could turn
this into an unbounded notification loop and prevent interface detach
from draining its link-state task.

Keep the cached PF state, but only publish link-up after iflib has
marked the VF running.  A subsequent admin pass publishes the cached
state after a successful initialization.

MFC after:      2 weeks

A PF reset indication leaves IAVF_STATE_RESET_PENDING set while the
VF recreates its AdminQ and negotiates new resources.  The ordinary
AdminQ task refuses to consume messages while that state is set.
Consequently, the first DISABLE_QUEUES reply after successful mailbox
rediscovery remains in the receive queue and initialization times out.
Later retries and manual interface restarts repeat the same cycle.

Clear the stale reset indication once VERSION and GET_VF_RESOURCES have
succeeded, before enabling interrupts and resuming normal virtchnl
requests.

MFC after:      2 weeks

This change introduces a variant of `pmap_invalidate_range` that uses
the fine-grained TLB invalidation instructions introduced by the
Svinval extension. These instructions allow for more efficient TLB
flushing on certain implementations. Under this new scheme,
`pmap_invalidate_range` was converted to an ifunc that selects the
appropriate variant during boot.

Event:  BSDCan 2026
Reviewed by:    markj, mhorne
Differential Revision:  https://reviews.freebsd.org/D57624

The VF link-status path can receive 2.5 and 5 Gb/s speed bits from
X550-family PFs, but media reporting has no cases for them.  The
bootverbose message also assumes every non-10-Gb/s link is 1 Gb/s.

Expose the corresponding ifmedia subtypes and derive the diagnostic
speed through the shared link-speed conversion helper.

MFC after:      2 weeks

X550-family devices provide clear-on-read counters for transmit and
receive Low Power Idle events.  Accumulate each register once in the
normal statistics poll and expose the monotonic totals below the eee
sysctl node.  Document the counters together with the existing EEE
control.

Obtained from:  Intel ix 3.4.39
MFC after:      2 weeks

10G-BX optics use paired wavelengths to carry 10 Gb/s Ethernet over a
single strand of single-mode fiber.  Their 10G compliance byte is
empty, so identify them from the SFF-8472 nominal signaling rate and
single-mode reach fields.

When an EEPROM also advertises 1G BASE-BX10, give the complete 10G
bitrate and reach signature precedence.  Otherwise retain FreeBSD's
permissive 1G-BX identification rather than requiring a nominal
1.3 GBd rate.

MFC after:      2 weeks
Relnotes:       yes

The 82599 and X540 share the global RSS redirection table between the
PF and its VFs.  Programming that table from the PF queue count
prevents a VF from using queue indices absent from the PF layout.  A
one-queue PF consequently directs every flow for a two- or four-queue
VF to queue zero.

Program at least four queue indices while SR-IOV is active.  Each pool
PSRTYPE.RQPL field masks the shared table to the queue subset available
to that function, so the PF can continue using fewer queues.

MFC after:      2 weeks

These are already available and having them defined helps keep the KASAN
atomic(9) interceptors uniform.

Reviewed by:    mhorne
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58680

According to Linux 5f1c3589b0f0, the X550 PHY classifier still matches
an alpha silicon ID, while the shared definitions contain the two
production IDs.  This can leave production hardware on the generic
probing path and issue unnecessary PHY queries.

MFC after:      2 weeks

The management_pkts_drpd sysctl was wired to MNGPTC, making it an
alias of management_pkts_txd, instead of MNGPDC.

MFC after:      3 days

Found with:           clang -Werror=assign-enum
Reviewed by:            markj
Differential Revision:  https://reviews.freebsd.org/D58710

Some devices take a little longer, and the spec doesn't really seem to
mandate a maximum.  The common path in usbd_req_set_address() has
already been bumped to 1s and I have a headset (Logitech H390) that does
need a little bit longer, so let's match it in xhci.

Reviewed by:    aokblast
Differential Revision:  https://reviews.freebsd.org/D58717

Added support for E835 adapters with post-quantum cryptographic (PQC)
algorithms in firmware/software signage and in SPDM attestation.

Signed-off-by: Pawel Sobczyk <pawel.sobczyk@intel.com>

Reviewed by:    Miłosz Linkiewicz <milosz.linkiewicz@intel.com>
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D57868

Add support for future client platform

MFC after:      1 week

Panther Point changed the reset value of CTRL_EXT.DPG_EN to enable
autonomous power gating. Clear it after hardware reset on Panther Point
and Nova Point controllers to prevent unexpected Tx/Rx hangs, packet
loss, or corruption.

MFC after:      1 week

pmc_capture_user_callchain() checks a PMC's runcount before walking
the user stack, but reads it without holding the spinlock that
protects it. hardclock() can run on the same CPU during the capture
and drop the runcount to zero in between, tripping the assertion and
panicking INVARIANTS kernels under load.

Move the check inside the existing spinlock, right where the code
already confirms the sample is still valid. No functional change on
kernels built without INVARIANTS.

Signed-off-by:  Andre Silva <andasilv@amd.com>
Reviewed by:    mhorne
MFC after:      1 week
Sponsored by:   AMD
Differential Revision:  https://reviews.freebsd.org/D58571

pmc_capture_user_callchain() asserts that TDP_CALLCHAIN is set on the
current thread, but PMC_UR samples never set that flag -- only PMC_HR
and PMC_SR do. That makes the assertion always fail for PMC_UR,
panicking INVARIANTS kernels as soon as pmcstat -U is used.

Skip the assertion for PMC_UR. No functional change on kernels built
without INVARIANTS.

Signed-off-by:  Andre Silva <andasilv@amd.com>
Reviewed by:    mhorne
MFC after:      1 week
Sponsored by:   AMD
Differential Revision:  https://reviews.freebsd.org/D58572

No functional change intended.

Sponsored by:   The FreeBSD Foundation
MFC after:      1 week

DPDK commit message

net/e1000/base: fix iterator type

Fix static analysis warning about comparison between types of
incompatible width, which might lead to an infinite loop due to
overflow.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Amir Avivi <amir.avivi@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (3d36053991)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix NVM data type in bit shift

There is a static analysis warning due to wrong data types being used
for NVM read data shifts. Fix it via explicit type cast.

Fixes: https://cgit.freebsd.org/src/commit/?id=38db3f7f50bd ("e1000: update base driver")
Cc: stable@dpdk.org

Signed-off-by: Przemyslaw Ciesielski <przemyslaw.ciesielski@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (b932270c66)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix possible variable overflow

Bits can be lost as temporary math is done on signed variables and the
result is assigned to an unsigned variable. Cast to u32 to force the
compiler to do operations on unsigned temporary variables.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Lukasz Czapnik <lukasz.czapnik@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (214cb0d7f1)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix NVM loop bounds and pointer access

Improve the NVM checksum routines by ensuring loop bounds are compared
at the correct integer width. Use array indexing instead of explicit
pointer arithmetic.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Menachem Fogel <menachem.fogel@intel.com>
Signed-off-by: Dima Ruinskiy <dima.ruinskiy@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (39fba42d04)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: improve NVM checksum handling

When reading NVM checksum, we may encounter the following scenarios:

- Checksum may be invalid, and can be updated
- Checksum may be invalid but cannot be updated because NVM is read-only

For the latter case, we should just ignore invalid checksum and not
attempt to update it.

Signed-off-by: Sasha Neftin <sasha.neftin@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (5241c17f0d)
MFC after:      2 weeks

Some transitional Tiger Lake systems shipped with an uninitialized
checksum word.  Accept that state while continuing to validate newer
read-only NVM images.

MFC after:      2 weeks

The shared semaphore helper accesses both the 82571 retry counter and
the I210 one-time-clear flag.  Those fields occupy overlapping members
of the device-specific union.  On 82571, incrementing the counter thus
enables the I210 recovery and clears SMBI after the first timeout.

Give 82571, generic 80003/82575, and I210/I211 users distinct acquire
paths.  Preserve the legacy peer-driver policy on 82571 and one-time
recovery on I210.

The separation follows the Intel e1000 base code in DPDK.

MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix semaphore timeout value

According to datasheet, software ownership of SWSM.SWESMBI bit should
not exceed 100ms. Current implementation caused incorrect timeout
counter values, where each iteration equals 50us delay. Because of that
driver was allowed to wait for semaphore even for 1.5s. This might
trigger DPC timeout.

This implementation hardcodes value to 2000, which multiplied by 50us,
gives 100ms of possible wait time.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Pawel Malinowski <pawel.malinowski@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (c8bcaf0f2a)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix unchecked return

Static analysis has detected a write that is not checked for errors,
leading to ignored error return value. Add a check.

Fixes: https://cgit.freebsd.org/src/commit/?id=edcdb3c5f71b ("e1000/base: fix link flap on 82579")
Cc: stable@dpdk.org

Signed-off-by: Dima Ruinskiy <dima.ruinskiy@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (b0b6b50c20)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: add LPI counters

Add new fields in structure to indicate if EEE LPI entries have been
observed on Tx and Rx path.

Signed-off-by: Sasha Neftin <sasha.neftin@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (2e8078ee69)
MFC after:      2 weeks

Accumulate the clear-on-read transmit and receive LPI event counters
on EEE capable PCH and I350 family devices.  Expose the 64-bit totals
under the per-device eee sysctl node.

MFC after:      2 weeks

sndcard_func is used as an ivar which passes around device info to the
PCM and MIDI children in snd_csa(4) and snd_emu10kx(4). Simplify this
and retire the need for sndcard_func, by 1) making an ivar only what
used to be stored in sndcard_func->varinfo, 2) replacing
sndcard_func->func with a child comparison, where needed, for instance
in csa_detach().

sndcard_func is harmless in reality, but there is no reason to have the
additional complexity. This way we also avoid the structure allocations.

Sponsored by:   The FreeBSD Foundation
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix reset for 82580

Fix setting device reset status bit in e1000_reset_hw_82580() function
for 82580 by first reading the register value, and then setting the
device reset bit.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Barbara Skobiej <barbara.skobiej@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (88a1eb79ef)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix MAC address hash bit shift

In e1000_hash_mc_addr_generic() the expression:

"mc_addr[4] >> 8 - bit_shift", right shifting "mc_addr[4]"

shift by more than 7 bits always yields zero, so hash becomes not so
different. Add initialization with bit_shift = 1, and add a loop
condition to ensure bit_shift will be always in [1..8] range.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (1749e662f6)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix data type in MAC hash

One of the bit shifts in MAC hash calculation triggers a static analysis
warning about a potential overflow. Fix the data type to avoid this.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Barbara Skobiej <barbara.skobiej@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (458734aaac)
MFC after:      2 weeks

The multicast hash bit can be 31.  Use an unsigned value so setting the
bit cannot shift a signed integer into its sign bit.

MFC after:      2 weeks

The i210 and i211 can occasionally fail to accept multicast table
writes, particularly while addresses are added and removed rapidly.
Read the table back and rewrite mismatches for up to three passes.

This prevents multicast reception from retaining stale filter state
while keeping the workaround limited to the affected controllers.

MFC after:      2 weeks

A reset NACK from a Linux PF means that the reset completed but no
permanent MAC address was assigned.  Treat that response as a
successful reset with a zero permanent address so attach can generate
a local address instead of retrying a live mailbox.

FreeBSD PFs also use a one-dword reset NACK while retained queues are
being sanitized.  Seed the otherwise unused request payload and accept
only the three-dword, zero-filled NACK used by Linux, preserving the
FreeBSD retry contract.

MFC after:      2 weeks

Return a PHY write failure immediately when disabling D0 low-power
link-up on 82571-family controllers.

MFC after:      2 weeks

The multicast hash bit can be 31.  Use an unsigned value so setting the
bit cannot shift a signed integer into its sign bit.

MFC after:      2 weeks

The multicast vector bit can be 31.  Use an unsigned value so setting
the bit cannot shift a signed integer into its sign bit.

MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix iterator type

Fix static analysis warning about comparison between types of
incompatible width, which might lead to an infinite loop due to
overflow.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Amir Avivi <amir.avivi@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (3d36053991)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix MAC address hash bit shift

In e1000_hash_mc_addr_generic() the expression:

"mc_addr[4] >> 8 - bit_shift", right shifting "mc_addr[4]"

shift by more than 7 bits always yields zero, so hash becomes not so
different. Add initialization with bit_shift = 1, and add a loop
condition to ensure bit_shift will be always in [1..8] range.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (1749e662f6)
MFC after:      2 weeks

DPDK commit message

net/e1000/base: fix data type in MAC hash

One of the bit shifts in MAC hash calculation triggers a static analysis
warning about a potential overflow. Fix the data type to avoid this.

Fixes: https://cgit.freebsd.org/src/commit/?id=af75078fece3 ("first public release")
Cc: stable@dpdk.org

Signed-off-by: Barbara Skobiej <barbara.skobiej@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Obtained from:  DPDK (458734aaac)
MFC after:      2 weeks

Do not modify a zero-initialized PHY control value when its preceding
read failed.  Leave the PHY unchanged when the void power helpers cannot
read its current state.

This follows the defensive checks added to the corresponding e1000
helpers.

MFC after:      2 weeks

A manageability VLAN can select bit 31 of its VFTA register.  Use an
unsigned value when constructing the register mask.

MFC after:      2 weeks

The PHY capability display examines all 32 bits of the firmware bitmap.
Use an unsigned value so examining bit 31 does not shift a signed
integer into its sign bit.

MFC after:      2 weeks

The driver already accumulates the clear-on-read transmit and receive
LPI event counters.  Expose the 64-bit totals under the per-device eee
sysctl node.

MFC after:      2 weeks

PHY identifier words are promoted to signed int when the cast is applied
after the shift.  Cast each 16-bit register value first so identifiers
with their high bit set are assembled as unsigned data.

MFC after:      2 weeks

The PHY identifier word is promoted to signed int when the cast is
applied after the shift.  Cast the 16-bit register value first so
identifiers with their high bit set are assembled as unsigned data.

MFC after:      2 weeks

The PHY identifier word is promoted to signed int when the cast is
applied after the shift.  Cast the 16-bit register value first so
identifiers with their high bit set are assembled as unsigned data.

MFC after:      2 weeks

No functional change intended.

Sponsored by:   The FreeBSD Foundation
MFC after:      2 weeks

I225 and I226 expose three programmable LED outputs.  Use LED1 for
adapter identification, following the convention in DPDK.  Preserve the
OEM configuration across identification requests.

Restore the OEM configuration before a device reset so an active led(4)
pattern cannot leave the output overridden across stop or detach.

The LED mode values follow the Intel I225 Software User Manual.

MFC after:      2 weeks

The generic LED on and off operations do not handle internal SerDes
media, leaving the led(4) device ineffective on my I210 fiber port.

Use the hardware blink operation for the on phase on internal SerDes.
The off phase restores the saved OEM LED configuration as before.

MFC after:      2 weeks

The debug routine advanced ring pointers as if rings were contiguous.
They are embedded in queue structures, so rings beyond queue zero had
the wrong stride.  The bogus queue index could cause an invalid MMIO
read and panic the machine.

Index the queue arrays first and then select the embedded ring.

MFC after:      2 weeks

The debug routine reads queue registers by queue index.  It also
advanced unused pointers to rings embedded in queue structures.  Those
pointers had the wrong stride and could proceed beyond the ring object.

Remove the unused pointer arithmetic.

MFC after:      2 weeks

Expose the physical port identification LED through /dev/led/ix*.
Save and restore the NVM-selected LEDCTL value around each request.
The X550 operations also clear their PHY manual override before the
register is restored.

Use the dedicated firmware port-identification command on E610.  Its
interface selects between firmware blinking and the original mode
rather than directly controlling LEDCTL.

Restore the normal indication before a device stop or reset.

MFC after:      2 weeks

Expose each physical port identification LED through /dev/led/ixl*.
Use the existing GPIO LED helpers for most devices and the PHY
provisioning interface for X710 10GBASE-T adapters.

Preserve and restore the original GPIO or PHY indication mode,
including before the interface is stopped.

MFC after:      2 weeks

I225 and I226 report uncorrectable internal memory errors through
ICR.FER and identify the affected region in PEIND.  Depending on the
region, hardware stops transmit or all PCIe and DMA traffic until the
port is reset and reinitialized.

Enable the fatal error interrupt and capture its read clear status in
the interrupt filter.  Mask the cause while an iflib reset is pending,
report the affected memory regions, and expose per region indication
counters.

PCIe region parity failures require a different recovery order from a
normal reset: assert DEV_RST, wait at least 3 ms, disable PCIe master
requests, clear PCIEERRSTS, and then reinitialize the port.  Follow that
sequence before entering the normal reset path and clear the remaining
LAN status afterward.

The I225/I226 PBECCSTS layout is unrelated to the PCH layout previously
copied into the igc headers.  Replace those unused definitions with the
I225/I226 memory error register definitions.

Hardware validation used an I225-IT revision 3 and a debug kernel that
wrote only the documented self-clearing injection bits.  It did not
synthesize interrupt or status state.

Coverage, notably DMA and Mgmt are not fully testable in my setup:
    Region  Observed hardware status          Result
    LAN     PEIND 0x1, LANPERRSTS 0x200       Reset and recovered
    PCIe    PEIND 0x4, PCIEERRSTS 0x8         Reset and recovered
    DMA     DRPARC injection read back zero   DFT-gated on test NIC
    Mgmt    Host debug strap unavailable      Not injectable

The repeated LAN and PCIe tests recovered without a panic or watchdog.
A PCIe-to-LAN sequence also verified that reset-time PEIND indications
are drained before FER is unmasked.

MFC after:      2 weeks
Sponsored by:   BBOX.io

I225 and I226 do not interrupt for corrected internal ECC errors.
Instead, the DMA packet buffer and PCIe memories expose sticky status
bits in PBECCSTS and PCIEECCSTS.

Sample these bits with the regular hardware statistics update, preserve
the PBECCSTS ECC enable state while clearing its RW1C indication, and
expose separate counters for the DMA packet buffer, PCIe transmit-data
memory, and PCIe retry buffer.

These counters represent observed indications rather than an exact error
count because multiple corrections between samples collapse into one
sticky status bit.

Hardware validation used an I225-IT (rev 3) and a debug kernel that
wrote only the documented self-clearing injection bits.  Each test
armed the injector, exercised the owning RAM with traffic, and compared
the corresponding counter before and after.

Coverage:
    Memory                Observed result
    DMA packet buffer     corrected_dma advanced once
    PCIe transmit data    corrected_pcie_tx_data advanced once
    PCIe retry buffer     No PCIe replay source; not exercised

The retry-buffer injector requires a real PCIe replay to read the
corrupted entry.  The test root port exposed AER and DPC reporting but
no protocol error injector, so ordinary traffic could not cover that
case.

MFC after:      2 weeks
Sponsored by:   BBOX.io

thunderbolt: Get NHI version number from caps

Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D49452

thunderbolt: Reset controllers

Reset routine for both v1.0 and v2.0 routes, chosen depending on version
reported in caps.

Reviewed by:    imp
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D49452

thunderbolt: Explicitly read NHI ISR0 register to clear it

This fixes and issue where Pink Sardine controllers were not receiving
interrupts for more than the first command sent on the ring.

Reviewed by:    emaste, imp
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D52862

Query the firmware for the LEDs on each physical port and expose
/dev/led/bnxt* only when alternate blinking is supported.  Configure
every LED in the advertised group for identification and restore its
default firmware state before a function reset.

This follows the DPDK and Linux bnxt HWRM identification paths.

Reviewed against: DPDK, Linux

Reviewed by:    Sumit Saxena <sumit.saxena@broadcom.com>
MFC after:      2 weeks
Sponsored by:   BBOX.io

Rewrite pmap_s1_invalidate_strided() to use range-based TLBI
instructions when they are when available.  This change can
significantly reduce the number of invalidation instructions
issued, leading to decreased system time.  (More details on the
decrease can be found in the review.)

Assisted-by:    Claude Code (Opus 5)
Reviewed by:    kib, markj
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58708

While testing an unrelated pmap change, D58708, that dramatically
reduces the number of TLBI instructions performed, and likely the
timing of unrelated events, I started seeing "Storing an invalid VFP
state" panics in vfp_save_state_common().  However, the origin of this
panic is elsewhere, in the else branch of sve_restore_state().
Specifically, my pmap change seems to have increased the likelihood that
the thread executing the else branch would be preempted by another
thread between the critical_exit() inside the else branch's call to
vfp_restore_state_common() and its own call to critical_enter().  Prior
to expanding the scope of the else branch's critical section, the MPASS
added by this change would fire, catching the problem at its source,
rather than later in vfp_save_state_common().

Assisted-by:    Claude Code (Opus 5)
Reviewed by:    kib, markj
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58723

A PCI function can provide a host bridge into a synthetic PCI domain.
Intel VMD does this: the host facing VMD function remains in its
original domain while the hidden Root Ports and endpoints appear in a
separate domain.  The VMD function's Device Control does not describe
an upstream link in that synthetic hierarchy.

The hierarchy wide cold pass incorrectly used the VMD function's MPS
to reprogram the hidden ports and their endpoints.  Stop both cold
reconciliation and runtime path walks at a PCI domain boundary.  The
real Root Ports within the VMD domain continue to reconcile their
endpoints normally.

Reviewed by:    imp
Tested by:      Michael Butler <imb@protected-networks.net>
Fixes:          https://cgit.freebsd.org/src/commit/?id=8e9fe9996a1f ("pci: Reconcile MPS before attaching PCIe devices")
MFC after:      6 days
Sponsored by:   BBOX.io
Differential Revision:  https://reviews.freebsd.org/D58837

Move the capability and quirk checks used by pcie_flr() into a public
side effect free helper.  This lets callers determine whether an FLR
can be attempted before quiescing a device or saving state.

The helper considers the advertised PCIe FLR capability and both the
enable and disable FLR quirks.

MFC after:      2 weeks
Sponsored by:   BBOX.io

E610 Hyper-V VFs use PCI configuration space communication instead of
the native PF/VF mailbox.  The generic E610 match currently attaches
native mailbox operations to those devices, and the imported Hyper-V
subdevice identifier is incorrect.

Correct the subdevice identifier to 0x00ff, as used by DPDK shared
ixgbe code, and reject that subtype until ixv has a complete Hyper-V
operations table.

MFC after:      1 week
Sponsored by:   BBOX.io

E610 VFs no longer report the actual PF link state and speed through
VFLINKS.  They can consequently report the default 10 Gb/s speed even
when the physical link uses another rate.

Negotiate mailbox API 1.6 on E610 and request the PF link state with its
three-dword operation.  Retain VFLINKS as the fallback when an older PF
rejects API 1.6.  Permit API 1.6 in the inherited xcast and queue
discovery helpers so negotiating the newer revision does not disable
existing operations.

Use GET_QUEUES to replace E610's one-queue fallback with the grant from
the PF.  The common path continues to use one iflib queue set per data
MSI-X vector and caps the result at two queue pairs.

Preserve mailbox transport errors so the driver can distinguish an
explicit PF NACK from a transient timeout.  A NACK means clear-to-send
state was lost and requires a VF reset.  Preserve the last confirmed
link state across brief transport failures and publish link down after
three consecutive failures.

Poll E610 link state every two seconds, matching Intel's ixgbevf service
timer, and phase VFs across the intervening iflib timer ticks.  This
avoids a mailbox polling herd when many VFs share a PF.  Media-status
queries return the cached state instead of starting another synchronous
exchange.  An admin interrupt caused by a mailbox reply only checks for
an unsolicited PF reset, preventing a request/reply interrupt loop.

Hardware validation on an E610 10GBASE-T PF exercised 63 VFs.  Each VF
negotiated API 1.6, two queue pairs, and three MSI-X vectors.  Phased
polling kept 31 active VFs idle, and all 63 recovered after a PF
down/up cycle without watchdogs.

Adapt the API 1.6 link-state operation from DPDK shared ixgbe code.  The
timeout and NACK distinction follows Intel ixgbevf 5.3.25.

MFC after:      2 weeks
Sponsored by:   Dirk-Willem van Gulik from Web Weaving (E610 hardware)
Sponsored by:   BBOX.io

E610 inherits the X550-family virtualization registers, anti-spoofing
controls, and malicious-driver operations, but the frontend does not
advertise SR-IOV and cannot negotiate the mailbox revision needed by
E610 VFs.

Initialize the X550-family PF/VF mailbox registers for E610 and use
PFVFLREC for its VF reset events, following DPDK shared ixgbe code.
Advertise the E610 SR-IOV capability, accept API 1.6 only on E610, carry
the existing xcast and queue operations forward to that revision, and
return the cached physical link speed and state with the three-dword
E610 operation.  Unsupported RSS and optional feature requests continue
to receive explicit failures.

SR-IOV activation also enables the existing X550-derived per-pool MDD
recovery path on E610.  Document the expanded protection and link-state
coverage.

Hardware validation created 63 VFs and rejected a 64th without flapping
the running PF.  Invalid TX and RX descriptor DMA independently asserted
the offender's WQBR bit, gated only that VF, preserved sibling traffic,
and recovered after the VF reset.  FreeBSD ixv, FreeBSD DPDK, Linux
ixgbevf, and Linux DPDK exercised the PF mailbox and data paths.

MFC after:      2 weeks
Relnotes:       yes
Sponsored by:   Dirk-Willem van Gulik from Web Weaving (E610 hardware)
Sponsored by:   BBOX.io

The VF statistics registers are free running and are not cleared on
read.  The existing code records attach time bases and pre-reset totals,
but never uses either when publishing counters.  It instead replaces
the low hardware bits directly, so counters can inherit pre-attach
traffic or jump backward after a reset.

Accumulate modular 32- and 36-bit deltas, following DPDK, while keeping
the software totals across planned resets.  Establish a fresh hardware
baseline after each successful reset and invalidate the sampling epoch
when mailbox state is lost.  Detect unsolicited PF resets explicitly so
a reset while link is down cannot be mistaken for counter wrap.

Remove the unused base and saved-reset bookkeeping.

On E610, packet and octet counters remained monotonic across a VF FLR
and a PF down/up cycle.  Traffic after each reset advanced both RX and
TX counters.

MFC after:      2 weeks
Sponsored by:   BBOX.io

X550 family devices provide a separate RSS key, redirection table,
and MRQC register for every VMDq pool.  With SR-IOV enabled, the
driver continued programming only the global RSS state and never
selected MRQC.MULTIPLE_RSS.  VF-local RSS programming was therefore
ineffective.

Enable multiple-RSS mode for X550, X552, X553, and E610.  Initialize
the PF pool's 64-entry key, redirection table, and RSS hash controls.
Leave each VF pool untouched so its driver retains ownership of its
RSS key and mapping.

E610 folds IPv6 extension-header traffic into its base RSS selectors
and reserves the legacy EX selector bits.  Translate those requested
hash types rather than programming reserved bits.

With two E610 VFs active and four PF queue sets, eight fixed TCP flows
distributed across all four PF receive queues.

E610 uses the same per-pool mode according to the E610 Datasheet,
sections 7.1.3.6.2 and 8.2.2.8.20-21.

MFC after:      2 weeks
Sponsored by:   Dirk-Willem van Gulik from Web Weaving (E610 hardware)
Sponsored by:   BBOX.io

PFVFRSSRK contains ten 32-bit RSS key words, numbered 0 through 9.
The previous inclusive range incorrectly ended at 10.

MFC after:      1 week
Sponsored by:   BBOX.io

e1000: Recover from PCH packet buffer ECC errors

PCH LAN controllers beginning with I217 report uncorrectable
packet buffer ECC errors through ICR.ECCER.  Descriptor memory errors
stop the MAC and require a reset before traffic can resume.

Enable the interrupt on the PCH generations whose shared code setup
enables packet buffer ECC.  Capture the read-clear PBECCSTS value in
the interrupt filter, mask ECCER while recovery is pending, and request
an iflib reset from the admin task.  Reenable the cause only after
hardware initialization succeeds.

Hardware validation used an I219-LM and the documented ICS.ECCER bit to
generate the fatal interrupt.  This synthesizes the interrupt cause but
does not corrupt packet buffer memory or alter its ECC byte counters.
Three injections in one boot each requested one reset and recovered
traffic without a panic or watchdog.  IMS.ECCER and PBECCSTS.ECC_ENABLE
remained set after every reset.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Report PCH packet buffer ECC statistics

PCH packet buffer ECC status contains read-clear byte counters for
corrected and uncorrected errors.  Sample them with the regular
hardware statistics update and account for the snapshot captured by
the fatal error interrupt path.

Expose the counters and the number of reset worthy interrupt
indications under dev.em.N.memory_errors.  Keeping the reset counter
separate also preserves evidence when another status reader wins the
read-clear race.

Hardware validation used an I219-LM.  Three documented ICS.ECCER
injections advanced fatal_resets from zero to three, exactly once per
reset.  corrected_packet_buffer and uncorrected_packet_buffer remained
zero, as expected because ICS does not inject a memory error or alter
PBECCSTS.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Recover from I210 and I211 memory errors

I210 and I211 report uncorrectable internal memory errors through
ICR.FER and identify the affected region in PEIND.  Depending on the
region, hardware stops transmit or all PCIe and DMA traffic until the
port is reset and reinitialized.

Enable FER and all regional indication masks.  Discard indication state
left by firmware before enabling reactions, capture the read-clear
status in the interrupt filter, and keep the cause masked while recovery
is pending.  Report the affected regions and expose per-region
indication counters.  Management-only errors remain under firmware
control.

PCIe region parity errors require a different recovery order from the
normal reset path.  Assert the port-local CTRL.RST bit, wait at least
3 ms, verify reset completion, disable master requests, clear
PCIEERRSTS, and then enter normal port reinitialization.  Do not use the
device-wide CTRL.DEV_RST sequence used by I225 and I226.

Hardware validation used an I210 revision 3 and the self-clearing
LANPERRINJ retransmit-buffer bit 9.  It injected a real parity error
without synthesizing interrupt or status state.
Three injections in one boot produced the following result each time:

    Observed hardware status          Result
    PEIND 0x1, LANPERRSTS 0x200       Reset and recovered

fatal_lan advanced exactly once per injection.  All tests completed
without a panic or watchdog, and FER and the LAN parity masks remained
enabled after every recovery.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Report corrected I210 and I211 ECC errors

I210 and I211 do not interrupt for corrected internal ECC errors.
Instead, the DMA packet-buffer and PCIe memories expose sticky status
bits in PBECCSTS and PCIEECCSTS.

Sample these bits with the regular hardware statistics update, preserve
the I210/I211 PBECCSTS enable state while clearing its RW1C indication,
and expose separate counters for the DMA packet buffer, PCIe transmit
data, and PCIe retry buffer.

The counters represent observed indications rather than exact error
counts because multiple corrections between samples collapse into one
sticky status bit.

Hardware validation used an I210 revision 3.  Unlike I225 and I226, the
published I210/I211 register definitions do not expose self-clearing
injectors for these corrected ECC memories.  The three counter sysctls
were present and remained zero under line-rate traffic and three fatal
LAN parity recoveries.  PBECCSTS.ECC_ENABLE remained set after every
reset.  Actual corrected-error accounting was therefore not injected.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Recover from I350 memory errors

I350 reports uncorrectable internal memory errors through ICR.FER and
identifies the affected region in PEIND.  Depending on the region and
memory, hardware stops transmit, receive, or all PCIe and DMA traffic
until the port is reset and reinitialized.

Enable FER and all regional indication masks.  Capture the read-clear
status in the interrupt filter.
Record the fatal PCIe, DMA, and LAN status registers, keep FER masked
while recovery is pending, and expose per-region indication counters.

Use the datasheet required port reset before master disable order for
PCIe parity errors.  Reset for PCIe, DMA, and traffic-affecting LAN
errors.  Statistics and VF-mailbox parity errors only require their
status to be discarded and cleared; management-memory recovery remains
under firmware control.

Validated on an I350 (8086:1521 revision 1).  Three software-set FER
interrupts each advanced the unknown-region counter once, requested a
single reset, restored carrier and traffic, and left FER rearmed without
a watchdog.  The software-set cause has no subordinate error status, so
region attribution and region-specific clearing remain datasheet-based.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Report corrected I350 ECC errors

I350 does not interrupt for corrected internal ECC errors.  Instead,
the PCIe, DMA, packet buffer, loopback, and management memories expose
sticky status bits in their region-specific status registers.

Sample those bits with the regular hardware statistics update, preserve
the RX and TX packet buffer ECC enable state while clearing RW1C
indications, and expose counters grouped by memory region.  Each counter
records observed indication bits rather than exact error counts because
repeated corrections between samples collapse into one sticky bit.

On an I350 (8086:1521 revision 1), the ECC enables remained set.  All
corrected-error status registers remained clear across boot, interface
down/up, three FER recovery resets, and bidirectional line-rate traffic.
The device has no documented corrected error injector.  Therefore, the
per-region paths were validated against the register definitions rather
than an injected SRAM error.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Recover from 82576 memory errors

82576 reports fatal and non-fatal internal memory errors through
ICR.FER and ICR.NFER and identifies the affected memory in its
native PEIND layout.  Fatal errors can stop transmit, receive, or
both until software resets and reinitializes the port.

Enable the controller-wide parity detector and implemented PEINDM
reaction bits after hardware initialization, while preserving unrelated
register state and omitting the absent IPsec memories on 82576NS.
Enable both interrupt causes and capture the read-clear PEIND register
in the interrupt filter.

Keep the causes masked while the iflib admin task owns the event.
Acknowledge non-fatal packet data errors without disrupting the port.
Request normal port reinitialization for FER, a fatal PEIND source, or
the memory hang indication.  Do not apply the later I210/I350 register
layout or their special PCIe parity reset order.

Hardware validation used a dual-port 82576EB revision 1.  Firmware
left PEINDM at its 0x80000000 default; initialization explicitly
programmed the parity-enable bit and produced 0xffffff07 on both ports.
An NFER during two-stream TCP sustained line rate without a reset,
watchdog, or carrier event.  FER on the linked and disconnected ports
each requested exactly one reset.  The linked port resumed the existing
TCP sessions after autonegotiation.  PEINDM and both interrupt causes
were restored after every reset.

The injections set the ICR causes without corrupting SRAM, so their
empty PEIND values deliberately exercised the unknown source path.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Report 82576 memory ECC errors

82576 exposes clear-on-read corrected error counters for RX, TX,
switch, IPsec, descriptor-handler, PCIe retry, PCIe write, and
MSI-X memories.  The packet and descriptor memories also count
uncorrectable errors.

Sample each status register exactly once from the regular hardware
statistics update and immediately before handling a memory-error
interrupt.  Group the counters by packet buffer, descriptor handler,
and PCIe region.  Skip the absent IPsec block on 82576NS.

PRBESTS and PMSIXESTS are shared by both LAN ports.  Attribute an
indication to whichever attached port samples the clear-on-read
register first so it is not counted twice.

Hardware validation used an 82576EB revision 1.  All nine implemented
status registers reported their ECC-enable bit set.  The sysctl
counters remained clear across interface lifecycle, two-stream
line-rate traffic, and NFER and FER cause injections.  Each reset
preserved the ECC enables while the driver restored PEINDM reactions.

ICR cause injection does not corrupt SRAM, and the only documented
data injector is specific to the IPsec packet buffer.  Exact counter
increments for the other memories were therefore validated against
the register definitions rather than an injected ECC error.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Recover from 82575 memory errors

82575 protects its packet buffer and receive and transmit descriptor
handlers with ECC.  Correctable errors are repaired in hardware.
Packet data errors are contained to the affected packet, while the
native RX_PBUR, TX_PBUR, RX_DHER, and TX_DHER interrupt causes report
unrecoverable packet buffer or descriptor handler state.  The affected
traffic direction remains stopped until software resets the port.

Enable the three ECC blocks and hardware memory error reaction after
queue and filter initialization.  Capture the clear-on-read status
registers in the interrupt filter and keep all four native causes masked
while the iflib admin task owns the event.

Request port reinitialization for every native PBUR or DHER cause.
Packet data errors that do not raise a native cause remain count-only
and do not disrupt the port.  The captured status registers provide
diagnostics and accounting but do not independently initiate recovery.

Hardware validation used an 82575EB revision 2 and the documented PBEEI,
RDHEEI, and TDHEEI injectors.  Correctable TX/RX packet data and
descriptor fetch/writeback errors preserved traffic.  Uncorrectable
TX/RX packet buffer header and descriptor fetch/writeback errors each
requested one reset, restored traffic, and rearmed every ECC control.
Repeated recovery produced no watchdogs.

MFC after:      2 weeks
Sponsored by:   BBOX.io

e1000: Report 82575 memory ECC errors

82575 exposes clear-on-read, saturating counters for corrected and
uncorrected errors in the packet buffer and the receive and transmit
descriptor handlers.

Sample all three status registers together from the regular hardware
statistics update.  When an unrecoverable event interrupts first, count
the values captured by the interrupt filter so the clear-on-read status
is not lost before the admin task handles it.

Expose packet buffer and descriptor handler counters under the existing
memory_errors sysctl node.

Hardware validation used an 82575EB revision 2 and the documented PBEEI,
RDHEEI, and TDHEEI injectors.  Correctable and uncorrectable TX/RX
packet-buffer errors and receive/transmit descriptor-handler errors
advanced the corresponding counters.  The controls and accounting
survived repeated recovery resets and an ordinary interface down/up.

MFC after:      2 weeks
Sponsored by:   BBOX.io

A media change can require a complete controller reset.  Resetting the
controller directly from the admin task leaves iflib rings, filters, and
interface state programmed for the pre-reset controller.

Request an iflib reset for every media change.  This already was done
when SR-IOV was active; use the same lifecycle for the ordinary PF case.

MFC after:      2 weeks
Sponsored by:   BBOX.io

The reset helper discards reset_hw and init_hw errors.  Runtime
initialization then continues programming rings and filters, and iflib
publishes the interface as running even though the controller did not
reach a usable state.  Initial attach similarly continues into NVM and
MAC setup after a failed reset.

Return errors from the reset helper.  Fail attach when the controller
cannot be reset or initialized, and report runtime failures through
iflib_init_failed() so iflib leaves the interface stopped.  Also stop
register accesses and report the error when a stop-path reset fails.

MFC after:      2 weeks
Sponsored by:   BBOX.io

The reset helper discards igc_reset_hw and igc_init_hw errors.  Runtime
initialization then continues programming rings and filters, and iflib
publishes the interface as running even though the controller did not
reach a usable state.  Initial attach similarly continues into NVM and
MAC setup after a failed reset.

Return errors from the reset helper.  Fail attach when the controller
cannot be reset or initialized, and report runtime failures through
iflib_init_failed() so iflib leaves the interface stopped.  Also stop
register accesses and report the error when a stop path reset fails.  A
later successful initialization completes pending fatal error cleanup
and re-arms FER.

Cache a requested MAC address before reset, but let init_hw program RAR0
after reset succeeds.  Let iflib perform its normal attach-post failure
cleanup instead of releasing the same driver resources from both layers,
and make queue cleanup idempotent.

MFC after:      2 weeks
Sponsored by:   BBOX.io

The shared code provides e1000_check_phy_82574() to recognize a PHY
hang from saturated receive error and idle error counters, but em(4)
never calls it.

Run the check from timer driven admin work.  Match Intel e1000e by
requiring two consecutive positive samples before requesting a full
iflib reset.

MFC after:      2 weeks
Sponsored by:   BBOX.io

The D100 device ID was defined and handled by aq_hw_capabilities(), but
had no entry in aq_vendor_info_array[], so the driver never probed it and
the card was left unattached.  Add the missing entry; the table lists the
fibre variant last within each group, so it follows D109 rather than
sorting numerically.

Signed-off-by: Nick Price <nprice@FreeBSD.org>
Accepted-by: adrian
Approved-by: adrian
(cherry picked from commit 4976b1d24abd6ff660e60041d1990b22c0bc2e5b)

aq_fw2x_thermal_arm() reached for a copper PHY register that the fibre
parts do not implement, so arming failed on every init and printed a
warning for a capability the hardware cannot have.  Return ENOTSUP when
the firmware does not advertise a temperature sensor, matching
aq_fw2x_get_temp(), and warn only for a genuine failure.

Signed-off-by: Nick Price <nprice@FreeBSD.org>
Accepted-by: adrian
Approved-by: adrian
(cherry picked from commit 3c7f1aa3b831431193106f8610b2142131d774f5)

Reviewed by:  mhorne
Differential Revision: https://reviews.freebsd.org/D57175

This change implements the core clknode methods for the SpacemiT K1
clock control units. These methods were used to implement drivers
for the APMU and PLL CCUs. The initial driver for the APMU CCU
only contains clock definitions for the SDHCI controller for now.

Differential Revision:  https://reviews.freebsd.org/D57176
Reviewed by:    mmel, mhorne, manu

Reviewed by:  mhorne
Differential Revision:  https://reviews.freebsd.org/D57178

style(9): sys/param.h, then sys/systm.h, then the remaining kernel
headers alphabetically.  imgact.h belongs before proc.h.

Reported by:    jhibbits
MFC after:      1 month
Reviewed by:    jhibbits
Differential Revision:  https://reviews.freebsd.org/D58884

T2, T1, and some pre-T1 Macs advertise a legacy PIO
range in the SMC ACPI _CRS alongside a live MMIO window, but the
silicon behind the PIO range is bogus.

Try MMIO first, validate via LDKN >= 2, fall back to PIO if that
fails or no MMIO resource is present. Drop "(T2)" from the backend
message since MMIO isn't T2-exclusive.

MFC: 1 week
Reviewed by:    ngie
Differential Revision:  https://reviews.freebsd.org/D58839

The problem observed on X550 adapters with 2.5 and 5 Gbps speeds
negotiation on some switches is not affecting E610 adapters.
Remove workaround, which omitted those speeds in the list
of initially advertised speeds and advertise all speeds
supported by adapter.

Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com>

Reviewed by:    kbowling
Tested by:      Mateusz Moga <mateusz.moga@intel.com>
MFC after:      1 week
Sponsored by:   Intel Corporation
Differential Revision:  https://reviews.freebsd.org/D57339

Two additional subdevice IDs were introduced
to distinguish between adapters with and without
manageability over USB support.

Signed-off-by: Krzysztof Galazka <krzysztof.galazka@intel.com>

Reviewed by:    erj
Tested by:      Mateusz Moga <mateusz.moga@intel.com>
MFC after:      1 week
Sponsored by:   Intel Corporation
Differential Revision:  https://reviews.freebsd.org/D57337

Update ieee80211_crypto_init_aad() to do what 802.11-2020 says -
only mask fc[0] bits 4-6 on data frames, not on management frames.
This (with other diffs to actually negotiate MFP and configure
ath(4) for MFP + software keys) allows the CCMP path to decrypt
CCMP MFP frames in the software path.

Differential Revision:  https://reviews.freebsd.org/D57799

There is no visible bug fixed as in current tree masks are the same.

Fixes:  https://cgit.freebsd.org/src/commit/?id=6883b120c53735ff1681ef96d257f376731f56b3

Add const-qualified versions of the NLA iteration helpers to allow
walking immutable netlink attribute buffers without discarding const
qualifiers.

This introduces NLA_NEXT_CONST(), _NLA_END_CONST(), and
NLA_FOREACH_CONST() in netlink_snl.h.

Signed-off-by: Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by : Google LLC (GSoC 2026)

Unloading if_ovpn while it's in use by other vnets causes
memory leaks and panics.
Fix this by reverting VNET_SYSUNINIT and adjusting the SI_SUB
initialization order.

Reviewed by:    markj
MFC after:      1 week
Differential Revision: https://reviews.freebsd.org/D54175

Revert pf_nl.h part of 017690e50913 and use new libsysdecode build glue
that parses enums.

Reviewed by:    kp, glebius
Differential Revision:  https://reviews.freebsd.org/D57866

The historical design of sockets is that on a re-connect the disconnect is
performed at the socket layer in soconnectat().  Since SMP times this is
known to be racy and the function has appopriate comment.  I missed that
in the recent change.  The pr_connect method should normally expect the
socket to be already disconnected, however should be able to handle a race
where socket is actually connected.  Convert the check that incorrectly
tried to handle normal path of re-connect into check that handles the
race.

Reported by:    markj
Fixes:  https://cgit.freebsd.org/src/commit/?id=ece716c5d34728a170f1dfe1b3389c267d6ddd1e

The ```int disable``` parameter is included in the bridge_stop function
signature but is not used in the function body.

I had noticed this when tracing the driver's path while learning more
about the ifnet library.

This parameter originally appeared when importing the driver from NetBSD.
However, the FreeBSD ifnet library no longer requires an if_stop function.
Meaning that the function signature can be changed to only contain needed
parameters for our bridge driver.

Discussed with: freebsd-net@ mailing list
Signed-off-by:  Acesp25 <acesp25@freebsd.org>
Reviewed by:    kp
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2290

A mistake from 90ea8e89d9b7 is that in6_pcblookup_internal() was skipped
for an inpcb that had unspecified local address.  This is incorrect, as
such inpcb could have already have a port set, and in_pcb_lport_dest()
shall not be called on such inpcb.  That could lead to creation of an
alised connection in the database.

This makes the function almost identical to in_pcbconnect().  While here,
fix minor bug of missing INP_ANONPORT.  This flag has no use in kernel,
but affects netstat(1) output in certain mode.

Fixes:  https://cgit.freebsd.org/src/commit/?id=90ea8e89d9b751e8b5ae90ef3397883b035788e5
Reviewed by:            pouria
Differential Revision:  https://reviews.freebsd.org/D57987

And make net.inet.ip.portrange.randomized boolean.

Reviewed by:            pouria, tuexen, markj
Differential Revision:  https://reviews.freebsd.org/D57291

Enable IFCAP_MEXTPG by default, which may bring performance benefits.
Allow it to be disabled, and when disabled assert that we do not receive
any mbufs with M_EXTPG set. This is useful for testing.
Default the tests to disabling MEXTPG support.

Reviewed by:    zlei
Sponsored by:   Rubicon Communications, LLC ("Netgate")
Differential Revision:  https://reviews.freebsd.org/D58054

The network layer must not pass unmapped (M_EXTPG) mbufs to if_output()
of network interfaces without IFCAP_MEXTPG. pf should convert these
mbufs by mb_unmapped_to_ext() for such interfaces but it didn't.
The problem had occurred on sendfile because sendfile system call
uses unmapped mbufs for the file data.

Reported by:    feld
Reviewed by:    kp, glebius
Differential Revision:  https://reviews.freebsd.org/D58021

Some nic drivers (including iflib) do not initialize if_hwassist
until after the interface is brought up.  If a lagg member is included
in a lagg when its not yet been brought up, that will cause lagg to see
if_hwassist=0 and will disable all checksum offload, etc, on the interface.
This is almost impossible to debug without kgdb or dtrace, as ifconfig does
not surface if_hwassist.

Fix this by re-calculating lagg caps (including if_hwassist) after adding a port.

I encountered this problem when I had a commented-out if_foo1=up entry in
my rc.conf that i neglected to uncomment when I was re-configuring a lagg.

Sponsored by: Netflix
Reviewed by: markj, zlei
Differential Revision: https://reviews.freebsd.org/D58062

RFC 4391 (IP over InfiniBand), section 9.3, lays the ND source/target
link-layer address option out as type, length (3), two reserved zero
octets, then the 20-octet IPoIB link-layer address.
The ND code assumed the Ethernet layout (RFC 4861, section 4.6.1)
everywhere and read/wrote the address directly after the option
header, i.e. two octets early.
The option-length sanity check computes 24 for both layouts for
a 20-octet address, so the mismatch was silent.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296585
Reviewed by:    adrian, pouria
Sponsored by:   VersatusHPC
Differential Revision: https://reviews.freebsd.org/D58096

From openssh-portable commit c14709356563.

Reported by:    cy

Fixes:        https://cgit.freebsd.org/src/commit/?id=e7d02be19d40063783d6b8f1ff2bc4c7170fd434
Signed-off-by:  Yusuke Ichiki <public@yusuke.pub>

PR:   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295434

Update nexthop flags with interface link status events and
instead of checking link status of interface for every packet
only check the reachability flag of the final nexthop.

Reviewed by:    glebius
Discussed with: markj
Differential Revision: https://reviews.freebsd.org/D57375

If a nhop gets an interface event, revalidate the nhops and
immediately try to recompile existing nexthop groups by
replacing unreachable nexthops with reachable ones.
If none are available, recompile them back to
their normal position in nexthop group slots.

Reviewed by:    glebius
Discussed with: markj
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D57389

Provide a name for SCTP sockets.

Fixes:          https://cgit.freebsd.org/src/commit/?id=8b2b62b49d88 ("sockstat: consolidate unix(4) protocols in the array of protocols")

Fix missing Include which is currently leaked through vnet code.

Reported by: bz
Fixes: https://cgit.freebsd.org/src/commit/?id=d05d1f256082 ("routing: Subscribe nhops to ifnet link events")
Differential Revision: https://reviews.freebsd.org/D57375

Per sys/conf/files this unit is not compiled for a NOIP kernel.

Reported by:  Alexander Sideropoulos <Alexander.Sideropoulos@netapp.com>
MFC after:      1 week

This is exactly the same as the second part of IPv4's change
136c5e17b61a1/D49153.

Make R-bit per RFC 6275 8.3 and P-bit per RFC 9762 7.1 in
Prefix Information option available to userland for future implementations.

RFC 9762 7.1: For each interface, the client MUST keep a list of every prefix
that was received from a PIO with the P flag set and currently has a non-zero
preferred lifetime.

Differential Revision: https://reviews.freebsd.org/D56207

- Early return when no new data is delivered
- Switching from PRR-CRB to PRR-SSRB only when both SND.UNA advances and no further loss is indicated.
- Accounting for sequence ranges SACKed before entering recovery in RecoverFS calculation.
- Force a fast retransmit upon entering recovery when prr_out is 0 AND SndCnt is 0.
- Set cwnd to ssthresh post recovery.

Obtained from: mohnishhemanthkumar_gmail.com
Reviewed by: rscheff, tuexen
Differential Revision: https://reviews.freebsd.org/D56535
MFC after: 3 months

Add FIB selection logic by introducing ifa_ifwithaddr_fib() to
support FIB-specific lookups. Then have ifa_ifwithaddr() wrap it
with RT_ALL_FIBS.
Also, do the same for ifa_ifwithaddr_check().

Reviewed by:    glebius, bnovkov
Differential Revision: https://reviews.freebsd.org/D58305

Free mbuf and increase IFCOUNTER_IERRORS if ip_ecn_egress() under
geneve_input_inherit() decides to drop the packet.

Reported by: Chris Jarrett-Davies of the OpenAI Codex Security Team
Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D58361

Sponsored by:   NetApp, Inc.
MFC after:      1 week
Reviewed By:    tuexen, #transport, markj
Differential Revision: https://reviews.freebsd.org/D58360

When a protocol-specific 'bind_all_fibs' tunable is set to 0, a
listening socket will only receive traffic originating from the FIB
it was bound to. However, there are no checks to determine whether
an address exists in the target FIB when binding the socket, which can
lead to a situation where a socket and the address it was bound to
belong to different FIBs.

Prevent this footgun by looking up the requested address in the current
FIB if 'bind_all_fibs' is active and returning an error if the address
does not exist.

Sponsored by:   Stormshield
Sponsored by:   Klara, Inc.
Differential Revision:  https://reviews.freebsd.org/D58281
Reviewed by:    glebius, pouria, markj
MFC after:      2 weeks

In practice, this is not possible, but we are adding it just to be safe.

Reported by:    markj

pf(4) currently ignores fragment direction (in vs. out)
in pf_frnode_compare() function.

Issue noticed and reported by Frank Denis

OK @bluhm

Obtained from:  OpenBSD, sashan <sashan@openbsd.org>, eaa2c80721
Sponsored by:   Rubicon Communications, LLC ("Netgate")

Outbound packet which matches rule with source limiter attached,
for example:

    source limiter "crash" id 1 entries 10000 limit 1000
    pass out from any to any source limiter "crash" keep state

triggers a NULL pointer dereference.

The issue was kindly reported and initial version of fix
submitted by SecBuddyF, Tencent KeenLab.

The submitted diff fixed the issue for failing look up by destination
address in outbound packet. dlg@ also pointed out the change should
be further improved so NULL pointer dereference is avoided when rule
uses nat-to/rdr-to option.

OK dlg@

Obtained from:  OpenBSD, sashan <sashan@openbsd.org>, f0f215c11e
Sponsored by:   Rubicon Communications, LLC ("Netgate")

The latest version of draft-ietf-tcpm-tcp-ghost-acks changed
a condition. This should make no substantial difference, but makei
it compliant to the latest version of the specification.

Reviewed by:            rscheff, Peter Lei
MFC after:              3 days
Sponsored by:           Netflix, Inc.
Differential Revision:  https://reviews.freebsd.org/D58411

During call to `icmp_verify_redirect_gateway()` ensure using
fib-aware source address selection function.

Reviewed by: glebius
Differential Revision: https://reviews.freebsd.org/D58409

When we drop the prefix lock to call nd6_prefix_offlink() or
nd6_prefix_onlink(), make sure to keep the correpsonding prefix
structure alive.  It is possible for a concurrent nd6_timer() to expire
the prefix while the lock is dropped.

Reported by:    Maik Muench of Secfault Security
Reviewed by:    pouria, zlei
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58423

inpcb: declare struct in_conninfo as a single declaration

This removes just one level of #define mess that is needed to reach into
an inpcbs IPv4 address.  And makes the declaration easier to read.  No
functional change.

Reviewed by:            markj
Differential Revision:  https://reviews.freebsd.org/D58273

libdtrace: Fix up translators after struct in_conninfo changes

Fixes:  https://cgit.freebsd.org/src/commit/?id=698402f4f97c ("inpcb: declare struct in_conninfo as a single declaration")

libdtrace: Fix up one more translator

Fixes:  https://cgit.freebsd.org/src/commit/?id=d8bcb13b79b4 ("libdtrace: Fix up translators after struct in_conninfo changes")

in_ifprimaryaddr() exists only to support IPv4 multicast usage. Since the
adoption of epoch tracking, ifa_ref() is no longer required in its body;
that was originally introduced by rwatson in 2009.

We could not use __deprecated1() from <sys/cdefs.h> anyway, as
IFP_TO_IA() is a macro, not a function.

Approved by:    glebius (2026-02-26)
Reviewed by:    adrian, glebius, pouria
Differential Revision:  D55344

IPv4 multicast currently has the big caveat that it depends on the first
assigned IPv4 address on an interface (the so-called "primary address").

in_ifprimaryaddr() only needs to be used by the following:
 - the 0.0.0.0 booting node input workaround in IGMPv1;
 - filtering out the node's own reports in IGMPv2;
 - preserving the source IP where an IGMPv3 report has been looped back;
 - inferring the default upstream IPv4 interface address for the
   IP_MULTICAST_IF socket option;
 - and inferring the source address during ip_output() for a multicast
   datagram where an interface has been explicitly specified by that option.

All of these uses mandate the use of IPv4 source address selection, but
FreeBSD does not yet (fully) implement this functionality.

Approved by:    glebius (2026-02-26)
Reviewed by:    adrian, glebius, pouria
Differential Revision:  D55345

If we have source address specified, try to find it by enumerating
ifas on specified fib.

Reviewed by: glebius
Differential Revision: https://reviews.freebsd.org/D58444

Revert UseBlocklist from SSHCFG_ALL to SSHCFG_GLOBAL (with
SSHCFG_COPY_NONE), ensuring it can only be set globally in sshd_config
rather than within conditional Match blocks, matching historical
behavior.

Reviewed by:    emaste
Fixes:          https://cgit.freebsd.org/src/commit/?id=bb5c77e9d281 ("OpenSSH: Update to 10.4p1")
Differential Revision:  https://reviews.freebsd.org/D58520

This may plug minor leaks which no-one has reported. The default IPv6 source
address selection policy list in FreeBSD is usually limited to 9 entries,
and can be readily inspected with ip6addrctl(8). The policy table is
however instantiated for each VNET.

The leak of a pol instance in delete_addrsel_policyent() was already
plugged by @ae in commit-id ecc5c73, so that change has not been merged.

Do not tear down the sxlocks as glebius has requested, and move the
addrsel_policyent{} declarations further up to avoid redundant forward
declarations as glebius requested for stylistic reasons.

Reviewed by:    ae, pouria
Sponsored by:   Cisco Systems, Inc.
Differential Revision: https://reviews.freebsd.org/D55599

When a TCP timer is stopped, t_timers[] is set to SBT_MAX. Adding the
corresponding t_precisions[], if it is not zero, would result in
overflows in tcp_timer_next(). To avoid this, skip stopped timers.

The problem was identified while debugging uperf by Lukas Book and
an initial patch was provided by him. The committed patch was
suggested by glebius.

The problem can be observed by running netstat -nxptcp and looking for
negative timer values and by observing very long running timers in
some cases.

Reported by:            Lukas Book <lkbook@outlook.de>
Reviewed by:            glebius
Differential Revision:  https://reviews.freebsd.org/D58484

nd6_ra_input() reads the IPv6 header pointer ip6 before m_pullup(), then
uses that pointer afterwards to set nd_ra.

When m_pullup() relocates the chain it frees the original first mbuf and
returns a new one, leaving ip6 dangling; the subsequent access may be a
use-after-free read.

The fix writes ip6 from the returned mbuf after m_pullup() inside the
conditional if.

Reviewed by: pouria
Differential Revision: https://reviews.freebsd.org/D58229

nd6_prefix_onlink will enter net_epoch when necessary.
Also, exit net_epoch earlier in nd6_prefix_onlink,
Because we acquired a reference to ifa, and we got our ifa
from the pr->ndpr_ifp, we don't need to stay under epoch.
While here, style it.

Reviewed by:    markj, glebius
Discussed with: zlei
Differential Revision: https://reviews.freebsd.org/D56129

Add the ability to select source ip address of outgoing packets
even when the source ip address is configured on another interface.
Also add this new rtnetlink attribute to manual.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=285422
Reviewed by:    glebius, ziaee (manpages)
Tested by:      ivy, Marek Zarychta <zarychtam@plan-b.pwste.edu.pl>
Relnotes:       yes
Differential Revision: https://reviews.freebsd.org/D58294

    Restrict the scope of the struct hc_metrics_lite to the kernel only.
    Update the naming to align with other kernel structures and add a tcp_ prefix.

Reviewed by:            glebius
MFC after:              2 weeks
Sponsored by:           NetApp, Inc.
Differential Revision:  https://reviews.freebsd.org/D58440

    When configuring the expire timeout to something short, make sure that the
    prune time runs at least at that interval. Similarly, when adjusting the
    prune interval up, ensure the expire timeout reflect that expected minimum
    time also. Finally, restart the callout timer so that the next pruning
    happens after the new, expected interval.

Reviewed By:            glebius
MFC after:              2 weeks
Sponsored by:           NetApp, Inc.
Differential Revision:  https://reviews.freebsd.org/D58424

Prior to commit 90ea8e89d9b751e8b5ae90ef3397883b035788e5, this was
handled by calling in6_pcbladdr().

Reported by:    syzkaller
Reviewed by:    pouria, glebius
Fixes:          https://cgit.freebsd.org/src/commit/?id=90ea8e89d9b7 ("netinet6: refactor in6_pcbconnect()")
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58518

Reported by:  Chris Jarrett-Davies of the OpenAI Codex Security Team
Reviewed by:    pouria, kp
Fixes:          https://cgit.freebsd.org/src/commit/?id=0361f165f219 ("ipsec: replace SECASVAR  mtx by rmlock")
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58521

Some devices remap the PF queues when entering or leaving SR-IOV. Add
opt-in PCI IOV helpers that hold the iflib context lock across the
complete stop, driver callback, and restart transaction.

Existing drivers continue to use the non-restarting helpers.

Sponsored by:   BBOX.io

Remove unused rnh_multipath and rib_algo_fixed members.
While here, convert rib_dying and rib_algo_init from uint32_t to bool.

Reviewed by: glebius
Differential Revision: https://reviews.freebsd.org/D58537

Add validation for unused parameter values in the gap between
VXLAN_PARAM_WITH_LOCAL_ADDR4 and VXLAN_PARAM_WITH_LOCAL_ADDR6 to prevent
panics.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297151
Reported by:    Robert Morris <rtm@lcs.mit.edu>
Reviewed by:    markj
MFC after:      3 days
Differential Revision: https://reviews.freebsd.org/D58552

if_gre(4): Fix races by changing initialization order and locks

Treat if_gre like any other network drivers during module
initialization by using SI_SUB_PROTO_IF.
Also, destroy cloned interfaces via a prison removal callback for
gre over udp.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275474
Reviewed by:    markj
Discussed with: glebius
Differential Revision: https://reviews.freebsd.org/D57669

if_gre(4): Fix link state announcement in SIOCDIFPHYADDR

Since we unlock gre before if_detach() and use slock in gre_clone_modify_nl()
there is no need to split if_link_state_change() out of gre_delete_tunnel().

Reported by: markj
Fixes: https://cgit.freebsd.org/src/commit/?id=a0d2e5ebaa2e ("if_gre(4): Fix races by changing initialization order and locks")

Migrate to new if_clone KPI and implement netlink support
for gif(4). Also break GIFSOPTS ioctl logic out of gif_ioctl.

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D57666

A RST segment can be sent in response to
(a) received segment or
(b) by the upper layer protocol.

The SEG.SEQ validation consists of two checks:
(1) the in-window check of SEG.SEQ and
(2) the exact match check of SEG.SEQ.

For the in-window check (1), the left edge of the window needs to be
based on tp->last_ack_sent to cover the delayed ACK case, whereas the
right edge needs to be based on tp->rcv_nxt + tp->rcv_wnd. This both
assumes that tp->rcv_wnd is not zero. For the special case of
tp->rcv_wnd being zero, add checks against tp->last_ack_sent for (a)
and on tp->rcv_nxt for (b). This applies to all TCP stacks.

When the exact match (2) of SEG.SEQ is performed, it should be based
on tp->last_ack_sent for (a) and on tp->rcv_nxt for (b). To cover both,
check for both. Add this only to the base stack, since the RACK and
BBR stacks already do this.

PR:                     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296594
Reviewed by:            rscheff
MFC after:              3 days
MFC to:                 stable/14
MFC to:                 stable/15
Sponsored by:           Netflix, Inc.
Differential Revision:  https://reviews.freebsd.org/D58594

This change is intended to address @glebius comments from the original D55663.
ip6_hdr_pseudo{} is referenced by certain OpenBSD OCF related components. I am
using __aligned(4) and not __packed as urged by the late Hans-Petter Selasky.
Use C99 types and style. We must eat the churn now cross-BSD compatibility
is "Fade to Black".

Put _Static_assert under #ifdef INVARIANTS to not disrupt regular compilation,
as this resides in a commonly included header file.

This brings xform_tcp.c into line with possible future OCF related imports.

Add support for allowing IPv4 multicast groups to be joined on IPv6 sockets,
as a number of applications began to rely on this over the years, despite it
only ever having been a convenience which appeared in Solaris & Linux over
the course of the 00s decade. It is limited to any-source joins (ASM).

To avoid further quibbling over the meaning of the term "undocumented" as it
applies to this change, I have chosen to use the wording "non-IETF-ratified
extension" in comments, with reference to the updated ip6(4) man page.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246

Comment updated only. No functional change.

It is unrealistic to expect that this feature will ever be resurrected from
the legacy KAME tree, given historical divergence, and that applications
which really need to consume all group state (e.g. proxies) will either
join on a per-group basis, or use link-layer mechanisms anyway. It was also
very poorly documented to begin with.

1. EFAULT was happening because sooptcopyin() from inp_join_group() was
seeing the user-space thread descriptor in the faked-up sockopt. So, do
not attempt a user copyin(); defer to C99 initialization nulling sopt_td
for us to force a KVA memcpy().

2. It seems necessary to byte-swap ipv6mr_multiaddr.s6_addr32[3] on amd64
for similar reasons as to how the user-space initialization needed for
passing an IPv4-mapped group address also requires byte-swapping of the
0x0000FFFF field for s6_addr32[2]; it is a direct assignment to a integer
member of a struct, NOT a memcpy().

3. The assignment to imr_interface within in6_v6_mreq_to_v4() was obfuscated
by a cast back to its own type due to use of the IA_SIN() macro. Elided.

With this change, the feature gap seems to be closed; tested with a simple
link-scope IPv4 group under 224.0.0.0/24 with an mlx5(4) SR-IOV VF in bhyve.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246
Differential Revision:  https://reviews.freebsd.org/D58590

cmd_securelevel is the securelevel at which the call should be denied.
pf (write) calls should be denied at level 3 or up (not at 2 or up as it
was), so increment these all by one.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296838
MFC after:      4 weeks
Sponsored by:   Rubicon Communications, LLC ("Netgate")
Differential Revision:  https://reviews.freebsd.org/D58377

pf assumes that network groups and network interfaces share a namespace
(that is, a name is unused, a group or an interface, never both a the
same time). Unfortunately this assumption was broken when interface
renaming was introduced.
Attempt to cope with this rather than panicking. Note that this is a
band-aid, not a full solution. The correct fix is for the network stack
to go back to enforcing a single namespace for groups and interfaces.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297220
Reported by:    Robert Morris
MFC after:      1 week
Sponsored by:   Rubicon Communications, LLC ("Netgate")

The SIOCGETSGCNT handler may be invoked in this scenario, and if no
router has initialized the lookup table, we'll have
mfct->mfchashtbl == NULL.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297148
Reported by:    Robert Morris
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation

This should assist setups that connect a lot of nodes to ipfw: and then
distribute traffic with ipfw(4) tablearg feature.

Reviewed by:            pouria
Differential Revision:  https://reviews.freebsd.org/D58547

iflib: restore TX watchdog functionality

Since f6afed726b00 the TX-hang check in iflib_timer() has required a
queue state other than IFLIB_QUEUE_IDLE, but nothing ever sets
IFLIB_QUEUE_WORKING, so IFLIB_QUEUE_HUNG has been unreachable ever
since: stalled TX queues are not detected, not reported, and not
reset - the TX watchdog of every iflib(4) driver has been dead code.

Instead of resurrecting the queue-state machine, detect the hang
directly.  A transmit queue is frozen while it holds descriptors the
hardware has not reported as completed and none were reclaimed over a
timer period.  Being frozen is not a fault: the hardware may defer
marking descriptors as completed indefinitely.  The check therefore
arms only when a frozen queue also takes on new work, while the link
is up, no pause frames were received and no doorbell is pending; and
it acts only after the queue has stayed frozen for
net.iflib.tx_watchdog_periods consecutive periods.  It then asks the
hardware through the driver's read-only credits peek
(isc_txd_credits_update with clear=false, the same call the mp_ring
can_drain callback makes routinely): if completions are ready but were
not harvested for this long, the completion interrupt went missing -
kick the queue's task instead of resetting; if the hardware reports
nothing although the queue kept receiving work, it is hung and the
existing watchdog reset machinery takes over.

Neither software counters alone nor mere persistence of unharvested
work can make this decision.  iflib reclaims lazily (up to
isc_tx_nsegments completed descriptors stay unharvested indefinitely)
and defers report-status requests, so "descriptors in use" and "no
cleaning progress" are normal states of an idle healthy queue.  And
hardware that coalesces completion reports (e.g. 8254x,
TXDCTL.WTHRESH) legitimately withholds the last one of a quiet queue
indefinitely, so a zero credits peek is a normal idle state, not a
hang indicator: arming on persistence alone reset healthy interfaces
on every traffic lull (field-tested on 82541PI).  Only growth across
frozen periods separates a wedged queue from a coalescing one.  The
threshold is a threshold in time, not in device work: a period is one
iflib_timer interval (hz/2 by default), so at the default of four
periods the verdict falls after roughly two seconds.  It was
calibrated from counter traces on that old and slow hardware, where
healthy coalescing always cleared within two periods; newer hardware
reports completions far sooner and leaves the frozen state earlier, so
the default needs no recalibration for more modern devices.  Setting
the sysctl to zero disables the check.

A queue whose link is down is never flagged - preserving what
f6afed726b00 fixed.

The new per-queue state goes into padding the transmit queue structure
already had, rather than next to the counters it is derived from: that
region is packed, so an insertion there would grow the structure.  What
is left of that padding is now spelled out instead of being implicit.
The size of the structure is unchanged on amd64, arm64, riscv64, i386
and armv7.

The IFLIB_QUEUE_* states no longer participate in the watchdog
decision; they will be removed in a followup commit.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=220997, https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239240
Fixes:          https://cgit.freebsd.org/src/commit/?id=f6afed726b00 ("iflib: Prevent watchdog from resetting idle queues")
Suggested by:   gallatin (mxge-style detection)
Reviewed by:    adrian, markj
MFC after:      1 month
Differential Revision:  https://reviews.freebsd.org/D58266
Assisted-by:    Claude Code (Fable 5, Opus 5)

iflib: remove the unused TX queue state machine

The previous commit stopped using ift_qstatus and the IFLIB_QUEUE_*
states for the TX watchdog decision, leaving only dead stores.
Remove the field, the states, and all assignments.  The byte the
field frees stays behind as explicit padding.  No functional change.

Reviewed by:    gallatin, markj
MFC after:      1 month
Differential Revision:  https://reviews.freebsd.org/D58282
Assisted-by:    Claude Code (Fable 5, Opus 5)

iflib: clear the deferred TX descriptor state when a queue is stopped

Stopping an interface frees the queued mbufs and zeroes a transmit
queue's descriptor accounting, but the three counters that track
descriptors deferred to a later doorbell write or report-status
request are not cleared there: they only reach zero when the code
that acts on them runs.  After a reset they therefore describe
descriptors that no longer exist, until enough new traffic flushes
them.

The consequences are small - one doorbell written from a stale
count, and a report-status request on the first packet after the
reset - but the state is simply wrong, and the transmit-hang check
in iflib_timer() reads one of them.

MFC after:      1 week
Assisted-by:    Claude Code (Opus 5)

A v6 raw socket may ask the kernel to validate the checksum of an
inbound packet.  If it does, and the validation fails, we discard the
packet, but this isn't really right: other raw sockets may wish to
receive a copy of the packet anyway.

Rework checksum handling to address this problem, and use a flag to
avoid computing the checksum more than once for a given packet.

Fixes:          https://cgit.freebsd.org/src/commit/?id=de2d47842e880281 ("SMR protection for inpcbs")
Reviewed by:    pouria, glebius
Reported by:    Yunzhi Ke
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58559

pfsync interfaces do not have ifp->if_inet6 set, so when we update the
MTU for those interfaces we panicked.

Add an explicit check for this. This should be temporary, until pfsync
is no longer a struct ifnet (as we've already done for pflog).

Reviewed by:    glebius
Sponsored by:   Rubicon Communications, LLC ("Netgate")
Differential Revision:  https://reviews.freebsd.org/D58701

pfsync packets were allocated with m_get2(), which can't return packets
larger than MJUMPAGESIZE. As a result 9k MTU pfsync interfaces simply didn't work.

Use m_get3(), which can allocate sufficiently large mbufs.

Extend the pfsync:bulk test case to provoke this problem.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297307
MFC after:      2 weeks
Sponsored by:   Rubicon Communications, LLC ("Netgate")

ifconfig: Add SR-IOV VF status output

- Adds SR-IOV VF status to the existing ifconfig "-v" output
- Adds ioctl command for reporting VF status info from drivers
- Adds support to iflib for drivers to handle this new ioctl
- Add support for ioctl in ixl(4)

Signed-off-by: Eric Joyner <erj@freebsd.org>

Relnotes:       yes
Differential Revision:  https://reviews.freebsd.org/D19647

iflib: Avoid locking for unsupported VF status queries

ifconfig -v requests SR-IOV VF status from every interface.  iflib
previously acquired the context lock before dispatching the request even
for VFs and drivers using the default unsupported method.  Mailbox work
on a VF could therefore delay the complete interface listing.

VF status describes the children of an SR-IOV PF.  Reject requests on
VF contexts and classes using the default method without taking the
context lock.  Keep the lock for actual PF status providers.

Fixes:  https://cgit.freebsd.org/src/commit/?id=1ccf543b21ef ("ifconfig: Add SR-IOV VF status output")

Honor RTEXT_FILTER_VF on RTM_GETLINK requests and expose the versioned
SR-IOV VF status through typed nested FreeBSD attributes.  Report
IFLA_NUM_VF with a successful requested query and preserve per-provider
errors in the status container.

Map the common nvlist schema to native integer, boolean, string, and
binary attributes.  Carry namespaced driver extensions as packed
versioned nvlists so adding a driver-specific field does not expand the
common netlink ABI.

Add SNL parsers, parser verification, a constructed nested-status test,
and an RTM_GETLINK test for an interface without SR-IOV support.
Document the query contract and every attribute.

When crypto_dispatch() or crypto_dispatch_async() returns non-zero,
the registered callback is never invoked. In both ovpn_transmit_to_peer()
and ovpn_udp_input(), if_ovpn.c did not free the cryptop request, release
the peer/sc reference count, or free the mbuf on dispatch failure.

This results in three simultaneous leaks per failed dispatch:
- crp allocated via crypto_getreq() is never freed
- peer->refcount (encrypt) or sc->refcount (decrypt) incremented but
  never decremented
- mbuf passed to crypto_use_mbuf() is never freed

The leaks are reachable under memory pressure when the OCF scheduler
returns ENOMEM from crypto_dispatch(). The registered callbacks
(ovpn_encrypt_tx_cb, ovpn_decrypt_rx_cb) correctly handle crp_etype
for crypto operation failures; this fix addresses the separate
dispatch-level failure path where no callback is invoked.

Found during code review following FreeBSD-SA-26:52.if_wg.

Reviewed by:    kp
Differential Revision:  https://reviews.freebsd.org/D58754

10GBase-BX uses paired wavelengths to carry both directions over a
single strand of single-mode fiber.  The optics must be paired so that
the transmit and receive wavelengths cross over.

MFC after:      2 weeks

Add missing TF_DISCONNECTED bit.

Reported by:    Hannes Elfert
Fixes:          https://cgit.freebsd.org/src/commit/?id=40dbb06fa73c ("inpcb: retire INP_DROPPED and in_pcbdrop()")

Initialize all three hashes (exact, wild, load balance group) with a per-
bucket lock.  Nothing changes for the packet lookup KPI - it still uses
SMR section for thread safety.  But connect(2) and bind(2) operations gain
parallelism now.

The main concept is that as we lookup inpcb database for editing, we are
accumulating bucket locks necessary to accomplish the operation.  Once all
lookups are complete and we are good to go, the inpcb is inserted (or
moved) and accumulated lock context is released.

Reviewed by:            markj
Differential Revision:  https://reviews.freebsd.org/D58131

When the first loop in inm_merge() hits an error, generally because it
hit some limit on the number of source filters for a multicast group,
inm_merge() tries to atomically roll back changes to the group source
filter list.

To roll back, it iterates over the global source filter list for the
multicast group, starting at the last entry that we updated ("nims").
But, if we have not yet updated any entries, this variable is
uninitialized.  Initialize it to NULL, so that RB_FOREACH_REVERSE_FROM
doesn't visit any source filters in this case.

All of the above applies to the v6 case.

Reported by:    Daniel Birtwhistle
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation

led(4) invokes driver callbacks while holding its mutex, including
from a callout.  iflib_led_func() cannot acquire the sleepable context
lock in those contexts without causing a lock-order reversal or sleeping
from the callout.

Record the latest requested state under the iflib state lock and
enqueue the existing per-device taskqueue.  The task can safely take
the context lock before invoking the driver.  Coalescing requests also
avoids accumulating stale blink transitions when hardware access is
slow.

Destroy the LED device before draining its task so no new callback can
race driver detach.

MFC after:      2 weeks

When a driver implements ifdi_led_func, have the framework create its
led(4) device after attach completes and the ifnet and context locks are
released.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246885
Reported by:    jlduran
Reviewed by:    markj
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D32389

A driver class may implement LED control even though the capability is
not available on every device or firmware version it supports.  Add an
optional capability method and consult it before creating the led(4)
device.  Default to supported so existing providers are unchanged.

This will be used by bnxt which blends PF and VF in the same driver.

MFC after:      2 weeks

Tables that have one element per protocol or address family were
previously sized by AF_MAX + 1 since AF_MAX was off by one.  Now that
AF_MAX has been corrected, we need to apply the opposite correction to
these tables.

Fixes:          https://cgit.freebsd.org/src/commit/?id=ddd850aa7720 ("sys/socket.h: Fix AF_MAX")
MFC after:      3 days
Sponsored by:   Klara, Inc.
Sponsored by:   NetApp, Inc.
Reviewed by:    pouria, kevans, glebius
Differential Revision:  https://reviews.freebsd.org/D58826

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58646

for SADB_UPDATE op SADB_X_EXT_NEW_ADDRESS_SRC/DST extensions, by
checking the sa_len matching the address family requirements before
doing the copy.

Also convert KEY_SETSECASIDX() and KEY_SETSECSPIDX() to functions and
apply the sa_len clamping there.

See https://github.com/0xdeadbeefnetwork/pfkey-sadb-overflow

PR:     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=297264
Tested by:      Wafa Hamzah <wafah@nvidia.com> (previous version)
Reviewed by:    markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58646

Noted and reviewed by:        markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58646

- Fixed memory leaks around m_dup() not freeing the original chain on
  failure. If we return ENOMEM, we are expected to have freed the
  chain, else the mbuf would be leaked. Also updated iflib_ether_pad()
  to follow the same structure.

- In iflib_parse_header()
    o Fixed a bug where the ip/ip6 and th pointers may point into a
      freed chain after m_pullup.  Those pointers must be reset to
      point into the new chain.

    o Eliminate ENXIO returns for non-TCP TSO sends (which would violate
      the mbuf ownership contract if they could happen). Since they
      cannot happen, I made them assertions instead.

- in iflib_ether_pad(), return ENOMEM after freeing mbuf, so that
  mp_ring knows it is free. An ENOBUFS error will cause the mp_ring
  path to retain the mbuf and retry

- in iflib_encap():
     o Fix a leak when bus_dmamap_load_mbuf_sg() returns ENOMEM
     o Fix a use-after-free in the mp_ring path when a driver using
       ktls frees an mbuf and returns ENOBUFS via iflib_encap()

After this change the expection from iflib_encap is that:

mp_ring: ENOBUFS can be returned only when we run out of descriptors
         (ENOBUFS causes mp_ring to retain the mbuf).

simple_tx: iflib_encap() always consumes the mbuf, regardless of the
           return

Note that iflib_debugnet_transmit(), like simple_tx, expects that
iflib_encap() always consumes mbufs.  This will be true after mp_ring
is removed, and its such a rare special case (overrunning the ring
during panic dumps) that I don't think its worth fixing in the
meantime.

Sponsored by: Netflix
Reviewed by: kbowling, sumit.saxena_broadcom.com
Differential Revision: https://reviews.freebsd.org/D58843
Fixes: https://cgit.freebsd.org/src/commit/?id=074ff8746388

lookup_route is only called for outgoing traffic, therefore
check nh_ifp index instead of nh_aifp as specified by RFC3542 sec 6.

Differential Revision: https://reviews.freebsd.org/D58544

Add six device-scoped fail(9) points at the registration milestones
needed to exercise each unwind path. An exact, runtime-only device
selector prevents unrelated iflib devices from consuming an armed point.

Mark the points non-sleepable because registration holds the ifnet and
context locks. Document one-shot operation and bus-address reprobe so a
failed attach can be recovered without another kernel build.

Reviewed by:    gallatin
MFC after:      2 weeks
Sponsored by:   BBOX.io
Differential Revision:  https://reviews.freebsd.org/D58722

iflib_device_deregister() sets IFC_IN_DETACH before removing the
interface, but a task which already passed its detach check can still
report a link change.  This can re-arm if_linktask after
ether_ifdetach() has drained it and leave work pending across queue
teardown.

Drain the entire private taskqueue before ether_ifdetach().  Drivers
may register their own link-related configuration tasks there, so
draining only the framework admin task leaves the same race for those
drivers.

MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58452

Co-authored-by: Andrew Gallatin <gallatin@FreeBSD.org>
Co-authored-by: Kevin Bowling <kbowling@FreeBSD.org>

Add an exact-device fail point immediately after the admin task checks
IFC_IN_DETACH. This makes the detach race reproducible without affecting
another interface.

Use a bounded delay to keep the task active while detach enters the
taskqueue drain.  Mark the point nonsleepable as a safety backstop, and
document a one-shot test for verifying that deregistration drains an
already-running task before ether_ifdetach().

Reviewed by:    gallatin, kgalazka
MFC after:      2 weeks
Sponsored by:   BBOX.io
Differential Revision:  https://reviews.freebsd.org/D58720

The VFLR task was initialized only from drivers MSI-X interrupt
assignment paths.  ixl's legacy interrupt handler can nevertheless defer
VFLR work, leaving an uninitialized task.  Even with MSI-X, the admin
interrupt was established before the task was initialized.

Initialize it alongside the other private tasks.  The existing detach
check and private-taskqueue drains then cover its lifecycle for every
interrupt mode and registration failure.

MFC after:      2 weeks
Sponsored by:   BBOX.io

When getting some baseline ALTQ numbers, I noticed that
if simlple_tx is enabled in kenv, we wind up re-setting the transmit
routine, but I neglected to actually clear ctx->ifc_sysctl_simple_tx.
That leads to many different panics as we run a mixture of mp_ring
and simple_tx.

Pointy-hat to: gallatin
Sponsored by: Netflix

Allow disabling RXCSUM and RXCSUM6 on an epair interface. If disabled,
epair unsets the mbuf flags that indicate a valid checksum when
transferring a packet from one epair end to the other. This gives a
user in a jail the power to control whether the user wants to use the
result of a previous validation (by a physical interface) or not.

Reviewed by:            kp, tuexen
MFC after:              1 month
MFC to:                 stable/15
Differential Revision:  https://reviews.freebsd.org/D58786

The same variable was used as a counter for an inner and out loop.
Add a new one for the inner loop.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=283934
Reported by:    crest at rlwinm.de

This tool supports two commands.  The list command outputs a summary
of injectable errors supported by the current system.  The inject
command injects the requested error.

Reviewed by:    gallatin, imp
Sponsored by:   Netflix
Differential Revision:  https://reviews.freebsd.org/D58026

LOG_CONS was OR'd into the facility argument instead of logopt, leaving
logopt as 0.  The correct call is openlog(ident, LOG_CONS, LOG_AUTH),
as shutdown(8) and init(8) already do.

PR:     https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296315
Signed-off-by: Ricardo Branco <rbranco@suse.de>
Reviewed by: imp, des
Pull Request: https://github.com/freebsd/freebsd-src/pull/2300

By default bhyve(8) creates a snapshot socket in
"/var/run/bhyve/" (BHYVE_RUN_DIR).
As this is a system directory not writable by users,
this does not work when bhyve(8) is being started
as a non-root user.

Address that by allowing to override this directory.
In bhyve(8) it is done by setting 'rundir' option
with '-o rundir=<path>'.

In bhyvectl(8) it is done with '--rundir=<path>'.

MFC after:              1 month
Reviewed by:            bcr (manpages), bnovkov
Sponsored by:           The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D57494

The download protocol calls download_data with FileOffset and
BufferLength of 0 first to start the download (no data yet
available). Calls it again with BufferLength == 0 and FileOffset the
size of the download (again, no data). It then starts calling with
BufferLength != 0 and FileOffset == 0 to start the download. The
heuristic I used to detect the start was wrong, so we'd allocate the
buffer twice. Fix that by being more explicit and not using the
heuristic that was bogus.

Fixes: https://cgit.freebsd.org/src/commit/?id=afee781523e4 ("loader.efi: Recognize new memdisk=<url> and memcd=<url> options")
Sponsored by:           Netflix
Differential Revision:  https://reviews.freebsd.org/D58068

The end address is the final byte in the array, not one byte past the
end of the array, so we need to add 1 to get the full length.

Fixes: https://cgit.freebsd.org/src/commit/?id=59219fc76a4b ("loader.efi: efiblk_memdisk_preload passes the VirtualDisks to FreeBSD")
Sponsored by: Netflix
Differential Revision:  https://reviews.freebsd.org/D58069

This code is simpler when we spell it the Unix way. Also, add sanity
checks to make sure the offset is where we think it is.

Fixes: https://cgit.freebsd.org/src/commit/?id=afee781523e4 ("loader.efi: Recognize new memdisk=<url> and memcd=<url> options")
Sponsored by: Netflix
Differential Revision:  https://reviews.freebsd.org/D58070

kldxref -m <file> will print the same data that the '-d' flag produces,
except restrict the output to one file. This should be the full path to
the file, and the directory name to process is omitted.

Sponsored by:           Netflix
Differential Revision:  https://reviews.freebsd.org/D57902

Reviewed by:  jfree, kib
MFC after:      3 weeks
Differential Revision:  https://reviews.freebsd.org/D58160

loader: Add xzfs, like gzipfs but with xz.

This is just like gzipfs or bzipfs, except done with the newer xz
program. This is off by default for the moment.

Sponsored by:           Netflix

loader: zstd based filesystem, zstdfs like gzipfs

Off by default.

Sponsored by:           Netflix

loader: Add forgotten xz.c and zstdfs.c

These were overlooked when I added compression support.

Fixes: https://cgit.freebsd.org/src/commit/?id=86d719ae68aa ("loader: Add xzfs, like gzipfs but with xz.")
Fixes: https://cgit.freebsd.org/src/commit/?id=c61ee49cd06a ("loader: zstd based filesystem, zstdfs like gzipfs")
Sponsored by:           Netflix

zonectl's Report Zones subcommand displays a tabular list of zones.  A
conventional zone's WP column is displayed as 0xffffffffffffffff , the
literal value that the HDD reports.  But that's too wide for the column,
causing the text to be misaligned.  It's also not really meaningful,
because the Write Pointer isn't really defined for a Conventional zone.
Change it to "-1" to fix the text misalignment.

MFC after:      2 weeks
Sponsored by:   ConnectWise
Reviewed by:    fuz
Differential Revision: https://reviews.freebsd.org/D57512

illumos smatch build is complaining:
    pci_nvme_parse_config() warn: 'sc->max_qentries' unsigned <= 0
    pci_nvme_parse_config() warn: 'sc->ioslots' unsigned <= 0

Because we are using atoi() to translate string to int, we need
to use int type variable for translation.

Reviewed by:    bnovkov
Differential Revision:  https://reviews.freebsd.org/D58213

Somehow, I wound up with space indents rather than tab indents, so fix
this.

Sponsored by:           Netflix

For devices like the rtw88, they will show up in `ifconfig -l` as
rtw880, rtw881, etc. We want to query the rtw88.0 and rtw88.1 sysctl
respectively, not rtw.880.

Chances are that there aren't more than 9 wlan devices using the same
driver. Use a better heuristic to get the device description.

Reviewed by:    bz
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation

Modify `blockif_open` to properly release a partially initialized
`blockif_ctxt` structure on error.

Differential Revision:  https://reviews.freebsd.org/D57887
Reviewed by:    novel, bnovkov, glebius
Tested by:      bnovkov
MFC after:      2 weeks

Since init become dynamically linked, reroot appeared to be broken
because init copies itself into a transient tmpfs mount to continue
controlling execution right after the reboot(REROOT) syscall.  Because
the binary is dynamically linked, it cannot be properly executed.

Provide a minimal static binary 'reroot_seed' embedded into the init as
byte stream, which performs what the 'init -r' did, namely, the second
phase reroot.

For the static build of init as part of the /rescue crunch, keep the
inline reroot code.

Reported and tested by: markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58164

pfr_add_table() does not set errno, it returns an error (now).
Read the error code from the return value so we display the correct
error message to the user.

Sponsored by:   Rubicon Communications, LLC ("Netgate")

These calls return an error value, they do not set errno. Check their
return values.

Sponsored by:   Rubicon Communications, LLC ("Netgate")

Pass such a section to the kernel using modinfo, otherwise link_elf.c
won't execute constructors for the file.  This is required for KASAN,
otherwise redzones for global buffers are not poisoned during boot.

Reviewed by:    kib
MFC after:      2 weeks
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58244

- Scope local variables properly to each function.
- Quote variables that should be treated as single words.
- Replace `${cmd}; if [ $? -eq 0 ]` with `if ${cmd}` for simplicity.

MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D57899

Add the option "oemstring" to allow setting the DMI type 11 ("OEM
Strings") SMBIOS structure. These are free-form strings, available for
any purpose, but can be especially useful to pass configuration,
secrets, and credential information into a Linux guest and consumed by
systemd.

MFC after:      1 month
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57516

When writing to a file, call fchmod() to ensure the file mode matches
the intended mode, which is 0444.  This was already done when replacing
an existing file, but not when creating a new file, which meant if the
process umask was 077, the resulting certificates and bundle would be
unreadable by unprivileged users.

MFC after:      1 week
Reviewed by:    des
Differential Revision:  https://reviews.freebsd.org/D58304

hwpmc: add regression tests for counting-PMC counter wraparound

Exercise a process-mode counting PMC whose accumulated count crosses,
or already exceeds, the range of the underlying hardware counter.
Before the previous commit, the first context switch after the
hardware counter wrapped panicked INVARIANTS kernels with "negative
increment" and silently corrupted the accumulated count on other
kernels.

The tests need a hardware counting event backed by a counter narrower
than 64 bits and skip where none is available (hwpmc(4) not loaded,
or a VM without a vPMU).

Reviewed by:            adrian
MFC after:              2 weeks
Assisted-by:            Claude Code (Fable 5)
Differential Revision:  https://reviews.freebsd.org/D58341

tests/sys/pmc: only build if MK_PMC != no

This unbreaks the build when pmc support is explicitly disabled via the
aforementioned build knob.

MFC after:      10 days
Fixes:          https://cgit.freebsd.org/src/commit/?id=2cfd82f74 ("hwpmc: add regression tests for ...")
Differential Revision:  https://reviews.freebsd.org/D58401

vt(4) does not (currently) support changing the video mode.  Report that
-i mode is not supported rather than printing an empty list.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207411
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58163

The max_qentries in pci_nvme_softc is uint16_t and
too large int may get truncated to invalid value.
While there, use local declarations for val.

Suggested by: Bill Sommerfeld
Reviewed by:    chuck
Differential Revision:  https://reviews.freebsd.org/D58293

Firmware on a test machine applied NX to non-code allocations, which
resulted in a fault when jumping to the trampoline.

Reviewed by:    kib
Tested by:      Jim Huang Chen <jim.chen.1827@gmail.com>
Sponsored by:   The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58383

MFC after:    1 week

Commit 9e1db51d4b5fc made nvmecontrol devlist get list of active
namespaces from the device instead of iterating through all possible
IDs.  The problem is that this request is not supported before NVMe
1.1, and in particular by Intel Optane 905P drives.  This change
reintroduces iteration for devices before NVMe 1.2.

Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D58010

Update fwget(8) to download wifi-firmware-mt76-kmod-mt7921, and
wifi-firmware-mt76-kmod-mt7925 firmware packages instead of the
no longer available mt792x version.
Add another PCI vendor to recognize ITTIM IDs for mt7921-based
MediaTek cards.

(bz reduced the license in the ittim file to an SPDX tag and updated
the commit message, given this is only half the work from the review)

Sponsored by:   The FreeBSD Foundation
MFC after:      3 days
Differential Revision:  https://reviews.freebsd.org/D57242

pmc: console configuration and table rendering for new PMC tools

Initializes the terminal rendering code used by the new pmc tools.  Then
provides a table abstraction for collecting, sorting and rendering
tables.  It provides pretty printed results with typed fields that print
several types used throughout the new PMC tools.  By default the fields
are formatted in engineering notation.

Sponsored by: Netflix
Reviewed by:    adrian, imp
Differential Revision:  https://reviews.freebsd.org/D57775

pmc: new pmc log processing framework

View is a class for building PMC log processing tools it is designed to
work with the new PMC record command that adds a header with additional
CPU information.  The new framework processes PMC logs about 2.5 times
faster and in about half the code as libpmcstat.

Sponsored by: Netflix
Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57776

pmc: pmc record command

The record command is designed around the idea of predefined studies.
While you can still select individual counters, the predefined studies
are meant to enable the best hardware options for a given generation.
It implements all of the base studies that I have built so far.

Sponsored by: Netflix
Reviewed by:    adrian
Differential Revision:  https://reviews.freebsd.org/D57777

pmc: pmc info command

Prints the log header including machine, cpu and kernel details along
with what counters were selected.

Sponsored by: Netflix
Reviewed by:    adrian, imp
Differential Revision:  https://reviews.freebsd.org/D57778

pmc: pmc frontend stall analysis based on IBS

The frontend command uses AMD IBS frontend events to analyze the major
sources of frontend stalls.  It displays a table breakind down the major
causes of front end stalls.  This is a simple demonstration of the tools
as you can use the filtering tools to limit the analysis to a subset of
the samples including filtering by fetch latencies.

Sponsored by: Netflix
Reviewed by:    adrian, imp
Differential Revision:  https://reviews.freebsd.org/D57779

pmc: enable the new pmc commands

This change hooks everything up to the pmc command and improves the
usage to document all functions.  There are a couple older commands that
are currently broken that I have hidden from the usage, but left in the
code for those using it.  I won't remove those until we have our
replacements upstreamed that depend on the AMD PMC multiplexing patches.

Sponsored by: Netflix
Reviewed by:    adrian, imp
Differential Revision:  https://reviews.freebsd.org/D57780

route(8): Add prefsrc option in netlink

Add prefsrc option that is frequently used on unnumbered interfaces
or L3 multi-homed network hosts.
This option uses RTA_PREFSRC.
Now you can add a static route by specifying the prefsrc option
with the loopback IP.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=285422
Reviewed by:    glebius
Differential Revision: https://reviews.freebsd.org/D58294

route(8): Add null check for prefsrc option

Reported by:    Shawn Webb <shawn.webb@hardenedbsd.org>, bms
Fixes:  https://cgit.freebsd.org/src/commit/?id=dd235f097af4 ("route(8): Add prefsrc option in netlink")

Modify the disk check to allow arbitrary files as the trailing argument
instead of requiring a live GEOM disk provider.

This enables modifying a boot0 binary file in-place before flashing it
to a disk via gpart bootcode, or using it directly as an argument to
mkimg's partition specification, as these tools cannot directly adjust
the parameters of the boot0 boot manager.

Reviewed by:    imp, jhb
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D57310

The need for this step is fading, now is mostly used to allow the
selection of just the two code partitions in the boot0 boot manager,
instead of the default of allowing all four MBR slices (the other two
being cfg and data, which cannot boot).

Reviewed by:    imp
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D57311

The test uses a fail point to inject a decryption error in OCF while
sending a ping across the tunnel.  The driver should then fail to
respond to the ping and increment the input error counter on the
interface.

Approved by:    so
Security:       FreeBSD-SA-26:52.if_wg
Security:       CVE-2026-58085
Reviewed by:    markj
Sponsored by:   Chelsio Communications

Reviewed by:  kib
Fixes:          https://cgit.freebsd.org/src/commit/?id=561991144e42 ("Remove Obj_Entry textsize member.")
Sponsored by:   AFRL, DARPA
Differential Revision:  https://reviews.freebsd.org/D58522

 - When cached response is available, actually use xid from one
instead of using its byte-swapped value for BIOS and 1 for UEFI.
 - If cached response is not available, generate pseudo-random xid,
since use of a constant may cause conflicts if two systems are
booting same time, and server sends responses as broadcast.
 - When cached response is available, skip DHCP DISCOVER/OFFER
and just send REQUEST to the DHCP server from the cached response.
We could skip this phase too and just use the cached response, but
we don't know whether firmware requested all of DHCP options we'd
like to get.

Tested on amd64 Supermicro X11DPI-NT for both BIOS and EFI, with
and without cached response packet.

Implement netlink support for gre in ifconfig

Differential Revision: https://reviews.freebsd.org/D55366

Transaction ID should persist only between OFFER and the following
REQUEST.  In all other cases it should change.

This implementation does not cover tunnel addresses.

Differential Revision: https://reviews.freebsd.org/D57667

Per RFC1717 section 5.1.3, the option length must be at least three.
Processing an undersized option would trigger a large out-of-bounds
write.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271910
Reported by:    Robert Morris
Reported by:    Décio Brandão (0xDBJ)
Reviewed by:    emaste
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58554

Some drivers do this routinely, e.g., FreeBSD's tpm20 does this every
time it sends a command in tpmcrb_transmit().  This causes the console
to fill up with messages.  Instead, only print a warning if the cancel
bit is set to one.

Reviewed by:    corvink
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D52425

Commit 505222d35fea removed a batch of code that this goto used to skip
around.

Reviewed by:    olce, kib, markj
Fixes:          https://cgit.freebsd.org/src/commit/?id=505222d35fea ("Implement the long-awaited module->file cache database.  A userland tool (kldxref(8)) keeps a cache of what modules and versions are inside what .ko files.  I have tested this on both Alpha and i386.")
Differential Revision:  https://reviews.freebsd.org/D58539

Previously all the 'goto out' statements after the image was loaded into
memory returned success rather than an error.  This is despite comments
indicating some of these conditions were in fact errors, and some of
these error conditions (such as missing PT_DYNAMIC) are treated as errors
in the kernel linker.

In addition, when failing to looking up the symbols for the linker
set, those cases returned failure leaking memory (though it's clear
from the original code from commit ca49b3342d1e that only the second
failure was intended to be an actual error).

To avoid more confusion, move the assignment of `ret` to just before
the `out` label so that `goto out` always returns an error.  This is a
more consistent pattern with other code in the tree that tends to use
labels for the error case.

Restructure some other code to avoid a few bogus errors.
Specifically, a symbol table is not required so don't treat lack of a
symbol table as an error.  Also, if the start symbol for the module
metadata linker set is not found, don't treat that as an error either.

Reviewed by:    kib
Differential Revision:  https://reviews.freebsd.org/D58540

All sorts of places in the ELF loading code assume that filesz <=
memsz, so check that explicitly up front.  The kernel already performs
this check for the PT_LOAD segments in the main binary and rtld in
imgact_elf.c.

Reviewed by:    jrtc27, kib
Differential Revision:  https://reviews.freebsd.org/D58541

Reviewed by:  jrtc27, kib
Differential Revision:  https://reviews.freebsd.org/D58543

Pass a single module name to load_kld for kbdmux and vkbd, allowing
bthidd_prestart to load both modules successfully.

Fixes:          https://cgit.freebsd.org/src/commit/?id=cfe1962a1925 (rc: Fix improper use of load_kld)
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation

Allowing nuageinit user scripts to run before these makes it possible to
customize official BASIC-CI and BASIC-CLOUDINIT FreeBSD images.

This was requested by KDE for their CI.

Approved by:    cperciva
Pull-Request:   https://ron-dev.freebsd.org/FreeBSD/src/pulls/60

Each byte of the address is represented by a pair of characters, so we
should be multiplying len by 2 when figuring out how much buffer space
we have.  Previously, a sufficiently large option could cause an
overflow of the global "result" buffer.

Reported by:    Joshua Rogers <joshua@joshua.hu>
Tested by:      Décio Brandão (0xDBJ)
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58555

This is in lieu of a full Kyua/ATF regression test, as this is an optional
feature that was beyond the scope of IETF's normative references for IPv6
multicast; support has been strictly on a best-effort basis.

Two new commands are added to mtest(8):
u mcast-addr ifname - join IPv4-mapped group on IPv6 socket
v mcast-addr ifname - leave IPv4-mapped group on IPv6 socket

Add an internal helper function __in6_v4_to_v4mapped() to perform the
converse of the IN6_IS_ADDR_V4MAPPED() check to support this use case.
Whilst __in6_v4_to_v4mapped() returns its first argument as a convenience,
avoid the temptation to dereference a pointer to that which we already hold.

Strictly the use of sockunion_t within mtest(8) more generally is a form
of controlled type punning (aliasing). Use a temporary as we overwrite
contents of su; the resultant write would overlap memory locations.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246
Differential Revision: https://reviews.freebsd.org/D58589

Fixes:                https://cgit.freebsd.org/src/commit/?id=7e2f38311e62 ("rtld-elf/rtld.c: apply clang-format")
Sponsored by:   Innovate UK

Populate stand/libsa/bootp.c's bootp_response global from the UEFI PXE
Base Code Protocol's cached DhcpAck, so bootp() can enter RFC 2131
INIT-REBOOT and skip DISCOVER/OFFER instead of running a fresh DHCP
transaction after the firmware has already done one.

It should give DHCP servers more information for proper responses.

During flag inconsistency report, we handle rai->rai_otherflg
as a bool, but the value is 0x40. Make it a simple number comparison.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295995
Reviewed by:    markj, Faraz Vahedi <kfv@kfv.io>
MFC after:      3 days
Differential Revision: https://reviews.freebsd.org/D58672

niov2 returns the number of entries in the iovec starting at offset
"offset".  Here we are unconditionally setting it to 1, which of course
isn't right.

Fixes:          https://cgit.freebsd.org/src/commit/?id=a28cf86c4171 ("bhyve/virtio: Rework iovec handling functions for efficiency and clarity")
Reported by:    Claude and Ada Logics
Reviewed by:    Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
MFC after:      1 week
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58625

bsdinstall: add a hardening knob for unprivileged kenv access

It makes sense.

Reviewed by:    zleei
Differental Revision:   https://reviews.freebsd.org/D57755

kern: fix oversight in security.bsd.unprivileged_kenv_read

It was intended that one could close the hole back in loader, but the
sysctl was actually not marked TUNABLE.  The hardening menu option thus
did nothing, because we wouldn't read the value from kenv.

Reported by:    markj
Fixes:  https://cgit.freebsd.org/src/commit/?id=6e81fbf5833d ("bsdinstall: add a hardening knob [...]")
Fixes:  https://cgit.freebsd.org/src/commit/?id=4fd518fcb2bb ("kern: add a security knob to disable [...]")

We can probaby consider these kernel bugs, in which case asserting is
not the most helpful thing we can do.  Let's emit the necessary details
to stderr and exit non-zero to aid debugging these without completely
blocking the ability to export all of the well-formed metrics.

Reviewed by:    rew
Differential Revision:  https://reviews.freebsd.org/D57983

PeriphID and CellID values are determined by macros which take an
index. They currently receive a bus offset which has a stride of 4 bytes.
This causes the ID1-3 registers to report incorrect values.
Scale the offset before passing it to the macro to fix this.

Tested with kvm-unit-tests/arm/pl031.

Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>

Reviewed by:    jrtc27
Fixes:          https://cgit.freebsd.org/src/commit/?id=014d7082a239 ("bhyve: Implement a PL031 RTC on arm64")
MFC after:      1 week
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2358
Closes:         https://github.com/freebsd/freebsd-src/pull/2358

Add -L to query the generic packed-nvlist IOV_GET_STATUS interface.
Report PF enable state and configured and total VF counts.  For each VF,
print its PCI address, newbus attachment, bound driver, and ppt state.

Retry size negotiation if the topology changes between ioctls and reject
malformed or incompatible status records.

Keep NIC-specific operational state in ifconfig -v; iovctl owns the
device-neutral PCI topology and applies to any SR-IOV device class.

Relnotes:       yes

- Prefix all structs with the struct keyword to avoid collisions between
  the types and variables with the same "name".
- Use `_` suffixed variables in initializers to distinguish input
  parameters from public members [1].

Resolve some trailing whitespace issues while here.

NOTE: this doesn't resolve the -pedantic issue reported by g++ with
`pmchdr_cpuidinfo::cpuid` about the field being a flexible array in an
otherwise empty struct.

1. I generally do this the other way around, i.e., suffix private/protected
   members with `_`, but these are public members in structs and I don't want
   to introduce a lot of churn in calling code.

Reported by:    g++14 with FreeBSD CI (powerpc64 tinderbox)
Fixes:  https://cgit.freebsd.org/src/commit/?id=ce6ab51f ("pmc: enable the new pmc commands")

This mutes a number of complains from g++ about needing specific headers
for functionality related to C strings and other function prototypes.

Reported by:    g++ 14

I need to do more work before references can be accepted in other
sections of the code. This was an unnecessary drive-by change that was
not tested in `make universe`.

Reported by:    CI
Fixes:  https://cgit.freebsd.org/src/commit/?id=fd809148 ("pmc(8): resolve -Wshadow issues")

nuageinit: adopt cloud-init disable_root semantics

disable_root now restricts root's authorized_keys instead of setting
PermitRootLogin.

Reported by:    np@

nuageinit: fix ssh_pwauth string handling

Treat "no"/"unchanged" correctly instead of any non-nil value as yes.

nuageinit: accept lock_passwd for users

Alias cloud-init lock_passwd key alongside locked.

nuageinit: support allow_public_ssh_keys

Skip importing datasource public keys when set to false.

Reviewed by:  markj
Differential Revision:  https://reviews.freebsd.org/D58636

bhyve: namescope virtio_msix to virtio.msix

The bhyve_config(5) variable `virtio_msix` is namescoped to
`virtio.msix`. Configurations that have the old variable will
automatically be mapped to the new one, with a warning message printed
out.

Relnotes:       yes
Reviewed by:    ziaee, markj
Differential Revision:  https://reviews.freebsd.org/D58390

fbsdrun_virtio_msix(): update virtio_msix to virtio.msix

Fixes:          https://cgit.freebsd.org/src/commit/?id=2d985d577d79 ("bhyve: namescope virtio_msix to virtio.msix")
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D58787

Allow "legacy" alongside "none" as a valid value for the ZFS mountpoint
property, matching zfsprops(7).

Reviewed by:    imp, markj
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58781

nuageinit_user_data_script references 'firstboot_freebsd_update'
and 'firstboot_pkg_upgrade', which are from Ports. In a default
base system test without sysutils/firstboot-freebsd-update and
sysutils/firstboot-pkg-upgrade, rcorder will warn on "unknown
provisions" to stderr, but is otherwise harmless.

Reviewed by:    arrowd
Fixes:          https://cgit.freebsd.org/src/commit/?id=16e47f317c4ce2be5fed530bf8a9af9f9bf55364
MFC after:      3 days
Sponsored by:   The FreeBSD Foundation

Co-authored-by:       Michael Osipov <michaelo@FreeBSD.org>
PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280487
Reviewed by:    kevans, michaelo
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D46313

rtld has always been built PIC since commit
7ca8e6a67068e8357e251bd3ea86253c8a751d59.  The stale #ifdef might
confuse a reader by thinking rtld can be built as non-PIC.

Reviewed by:    kib
Sponsored by:   AFRL, DARPA
Differential Revision:  https://reviews.freebsd.org/D58623

The name of the script and the name used internally for rc.conf differ,
as such the hardcoded disabling of service jails for the didn't work.
Fix by using the correct name.
Fixes:          https://cgit.freebsd.org/src/commit/?id=f99f0ee14e3af rc.d: add a service jails config to all base system services

We have ports and basesystem services, where the internal name and the
filename differ. While the documentation recommends to keep them in
sync, the reality is different. For service jails use the basename of
the service filename.
Fixes:          https://cgit.freebsd.org/src/commit/?id=2efbd48 rc: add service jails framework
Suggested by:   joneum
MFC after:      1 week
MFC to:         stable/15

nfsd: Update the rc.d script for RDMA for the nfsd service

Commit 7144a1d58c5c added the hooks for the nfsrdma.ko module.
Once loaded, this module adds RDMA support to the nfsd.

This patch adds a few lines to /etc/rc.d/nfsd, so that
nfs_server_rdma_enable="YES"
in your /etc/rc.conf will load nfsrdma.ko, so that RDMA
service is enabled.
It also supports
nfs_server_rdma_listen="port#"
so that the default of 20490 can be overridden in /etc/rc.conf.

At this available as time, the nfsrdma.ko module is an unofficial port,
since it was developed by Vinicius Ferrao <ferrao@versatushpc.com.br>
using generative AI.
As soon as it is available, it will be announced on
freebsd-current@freebsd.org.

Suggested by:   Vinicius Ferrao <versatushpc.com.br>
MFC after:      1 month

rc.conf: Fix the default NFS-over-RDMA port number

The default for nfs_server_rdma_listen transposed two digits: 20490
instead of 20049, the IANA-assigned port for NFS-over-RDMA.

Fixes:  https://cgit.freebsd.org/src/commit/?id=471e14267bea ("nfsd: Update the rc.d script for RDMA for the nfsd service")
MFC after:      1 month
Sponsored by:   VersatusHPC
Pull Request:   #2371
Signed-off-by:  Vinícius Ferrão <ferrao@versatushpc.com.br>

Currently, sending SIGTERM to the bhyve process triggers ACPI poweroff
for a VM. However, when running bhyve in monitor mode (-M), there are
two processes: the monitor process and the actual VM process. Sending
SIGTERM to the VM process works as before -- it powers off the VM.
But sending SIGTERM to the monitor process just kills the monitor
process, leaving the stale VM process running.

Fix that by creating a pipe between these two processes. The child
process uses the pipe to detect when the monitor goes away, and exits
automatically.

MFC after:              2 weeks
Reviewed by:            markj
Sponsored by:           The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58788

bsdconfig: add datetime module for live system clock

Provide bsdconfig datetime (menu), date, and time to set the running
system clock via dialog(1)/bsddialog(1) --calendar and --timebox with
mustberoot.  Unlike bsdinstall's time helper, this operates on the live
system rather than a CHROOT-staged install target, and calls
adjkerntz(8) after a successful change so the CMOS stays in sync.

Reviewed by:    bcr
Differential Revision:  https://reviews.freebsd.org/D58487

mtree: usr: add missing bsdconfig datetime directories

8cfe06ee4415 installs into 085.datetime and share/bsdconfig/datetime,
but those paths were not in BSD.usr.dist. In-place installworld then
fails when install(1) cannot create the destination.

Fixes:  https://cgit.freebsd.org/src/commit/?id=8cfe06ee4415 ("bsdconfig: add datetime module for live system clock")

Merge implementation of "traceroute -e" to traceroute6 for TCP/UDP/SCTP.

MFC-after:      2 weeks

On ZAC drives, conventional zones conventionally report a write pointer
LBA of 0xffffffffffff. This field is 48 bits wide, unlike ZBC's 64 bits.

Recognize both ZAC and ZBC all-ones behaviour in the WRITE POINTER LBA
field to indicate non-valid information.

Tested by:      fuz
Discussed with: fuz, asomers, ken
Fixes:          https://cgit.freebsd.org/src/commit/?id=4735ef6196bc ("zonectl: display conventional zones better")
MFC after:      2 weeks
Sponsored by:   Google Summer Of Code 2026
Reviewed by:    asomers
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2345

Fixes:        https://cgit.freebsd.org/src/commit/?id=255538cd906045095d0c2113ae6c4731ce36c0cf
Differential Revision: https://reviews.freebsd.org/D57850
Reviewed by:    adrian

MFC after:    1 week
Fixes:          https://cgit.freebsd.org/src/commit/?id=c3276e02beab ("sockets: make shutdown(2) how argument a enum")
Reviewed by:    glebius
Differential Revision:  https://reviews.freebsd.org/D57915

PR:   https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296410
Submitted by:   Tomas Vondra <tomas@vondra.me>
MFC after:      1 week

libsysdecode: decode Generic Netlink controller messages

Decode Generic Netlink controller (GENL_ID_CTRL) messages in
Netlink payloads. Display the Generic Netlink header along with
the CTRL_CMD_GETFAMILY attributes, including the family ID and
family name.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Reviewed by:    kp
Sponsored-by:   Google LLC (GSoC 2026)

libsysdecode: cache Generic Netlink family IDs

Record Generic Netlink family IDs learned from
CTRL_CMD_GETFAMILY responses and use them to decode subsequent
Generic Netlink messages using symbolic family names instead of
numeric IDs.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Reviewed by:    kp
Sponsored-by:   Google LLC (GSoC 2026)

libsysdecode: decode PF Generic Netlink commands

Decode the Generic Netlink command header for messages
belonging to the PF Generic Netlink family. Display the
command name using the PF Generic Netlink command decoder.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Reviewed by:    kp
Sponsored-by:   Google LLC (GSoC 2026)

libsysdecode: add attribute parsing for PFNL_CMD_GETRULES

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed by:    kp

libsysdecode: use decoder table for PF netlink commands

Introduce a PF netlink command decoder table mapping PFNL commands to
their attribute decoder sets. This replaces the existing switch-based
dispatch and makes it easier to add support for additional PF netlink
commands.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed by:    kp

libsysdecode: avoid extra commas for undecoded netlink attributes

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed by:    kp

libsysdecode: verify decoder tables are sorted

Add assertions to validate decoder table ordering required by
binary search.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed by:    kp

libc/resolv: Drop Solaris 2 compatibility

MFC after:      1 week
Reviewed by:    kevans, markj
Differential Revision:  https://reviews.freebsd.org/D57922

libc/resolv: Refactor the option parser

Start the loop by finding the end of the option name, the name-value
separator (if any), and the end of the option.  Use those pointers to
simplify matching the option name and parsing the option value, and
validate option names and values more strictly.  This means that:

* We no longer accept trailing garbage in an option name or value.  For
  instance, we would previously interpret “edns0123” as “edns0” and
  “timeout:3xyz” as “timeout:3”.  This was actually quite lucky because
  we also failed to recognize the newline at the end of the option line
  as a whitespace character.

* For options that take a numerical argument, we would previously accept
  negative values and treat non-numerical arguments as 0, while large
  numerical arguments would be capped to the option's maximum permitted
  value.  Now, any failure to parse the argument, including overflow,
  results in the option being left unchanged.

MFC after:      1 week
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57923

libc/resolv: Refactor the configuration parser

This was previously all a single loop in res_init(), apart from option
parsing which we cleaned up in a previous commit.  Break it out into
separate functions for reading the configuration line by line, setting
the default domain, setting the search list, and adding a nameserver
to the nameserver list.  Sprinkle bounds checks and code comments all
around.

The sortlist code, which has been disabled for the past 20 years, will
be dealt with in a separate commit.

MFC after:      1 week
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57924

libc/resolv: Reimplement the sortlist parser

When we switched from the BIND4 resolver to the BIND9 resolver, the
sortlist parser was inadvertently disabled due to a missing #define, and
nobody seemed to notice.  The sorting code remained enabled in the
resolver, but there was no way to set a sort order.

Reimplement the sortlist parser, but correctly, and update the manual
accordingly.  The new parser accepts IPv4 and IPv6 addresses with or
without a mask or prefix length, just like the old one, except IPv6
support was a bit wonky in the original code.

Fixes:          https://cgit.freebsd.org/src/commit/?id=5342d17f09a8 ("Update the resolver in libc to BIND9's one.")
Relnotes:       yes
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57925

libc/resolv: Add no-debug and no-rotate options

These are simply the reverse of the debug and rotate options.

Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D57926

Reviewed by:  markj
Differential Revision:  https://reviews.freebsd.org/D57927

getfacl / acl_to_text() incorrectly prints uid/gid numbers as signed integers.
This causes uid / gid numbers larger than 2G (2147483648) to print as
negative numbers.
The libc acl_from_text() function does not handle negative numbers.
This diff adds a backwards compatiblity fix to allow negative numbers...

Reviewed by:    rmacklem
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D57180

While here, clean up and simplify the existing code.

MFC after:      1 week
Reviewed by:    glebius, jhb
Differential Revision:  https://reviews.freebsd.org/D57993

Reviewed by:  markj
Tested by:      pho
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D57124

Reviewed by:  markj
Tested by:      pho
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D57163

libfetch: Overhaul socket read / write

* Make fetch_ssl_read() and fetch_ssl_write() behave more like read(2)
  and write(2), and drop fetch_socket_read() in favor of read(2).

* Don't request POLLERR, it's implied.

* Don't needlessly set errno, it's relatively costly.

* Always check for EAGAIN from writev(2), otherwise we will abort on a
  short write instead of proceeding to poll(2).

* Always check for EAGAIN from poll(2) even though it can't happen on
  FreeBSD; POSIX says it can, and it might in the future.

* Rewrite fetch_read() and fetch_writev() to be more similar to each
  other.  The main difference is that a partial read is treated as
  success while a partial write is treated as failure.

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296316
MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57906

libfetch: Add read buffering

Previously, we would read FTP control connection messages and HTTP
reponse headers one character at a time.  Now, we read as much as will
fit in our buffer and look for a newline.  If there is data left over,
it will be reused by the next fetch_getln() call.  This also requires
the addition of a fetch_bufread() which takes the buffer into account,
otherwise the start of the HTTP response body will be stuck in the
buffer after we read the last line of the header.

This should noticeably improve HTTP performance, especially for small
transfers.

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57907

libfetch: Apply timeout to connection attempts

Mark the socket non-blocking before connecting and poll for completion,
applying fetchTimeout if set.

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57909

Make fetch_ref() an inline and provide a fetch_deref().

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D57944

Reduce the amount of copying we do when performing buffered reads.

MFC after:      1 week
Reviewed by:    op
Differential Revision:  https://reviews.freebsd.org/D58113

Reviewed by:  zlei, vmaffione
Obtained from:  https://github.com/luigirizzo/netmap/commit/b52a2bcae35e56548acfb0849b248a1e4b0c0c3b
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58150

Reviewed by:  zlei, vmaffione
Obtained from:  https://github.com/luigirizzo/netmap/commit/7d9177ed9a121e66bf4eaa0acb5d574e408297da
MFC after:      2 weeks
Differential Revision:  https://reviews.freebsd.org/D58151

Logic prior to this change would incorrectly try linking when MK_CDDL != no,
instead of MK_CTF != no, which could result in the library and the tests being
broken if/when MK_CTF == no and MK_CDDL != no (an uncommon, but possible
combination with today's build knobs).

This change updates the conditional to correctly track the value of MK_CTF, which
in turn is properly toggled to no if/when MK_CDDL == no as it's a dependent build
knob.

This [niche] build bug has been present in FreeBSD since 2014.

MFC after:      1 week

* Get rid of the pointless LOCALBASE_CTL_LEN mechanism

* Apply minimal normalization to the paths obtained from the environment
  or sysctl variable

* Turn the manual page into a manual page

* Add tests

MFC after:      1 week
Reviewed by:    se
Differential Revision:  https://reviews.freebsd.org/D58362

MFC after:    1 week
Fixes:          https://cgit.freebsd.org/src/commit/?id=2a5e58c59694 ("procdesc: add NOTE_PDSIGCHLD")
Reviewed by:    kib
Differential Revision:  https://reviews.freebsd.org/D58388

Introduce a generic Netlink attribute decoding framework based on
attribute decoder tables. The framework supports decoding primitive
attribute types as well as nested attributes and can be reused by
different Generic Netlink families.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed-by:    kp
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2337

Sponsored by:   The FreeBSD Foundation

Sponsored by:   The FreeBSD Foundation

Replace <sysdecode.h> with "sysdecode.h" so local builds use the in-tree header
in lib/libsysdecode instead of a stale installed copy in /usr/include, avoiding
build failures after updating sysdecode.h.

Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)
Reviewed by:    kp
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2338

Change sysdecode_nlm_flag() to decode Netlink message flags as a
bitmask instead of looking up a single flag value. This correctly
prints combinations of NLM_F_* flags while preserving any unknown
bits in hexadecimal.

Reported by:    androvonx95 <androvonx95@tutamail.com>
Reviewed by:    kp
Signed-off-by:  Ishan Agrawal <iagrawal9990@gmail.com>
Sponsored-by:   Google LLC (GSoC 2026)

Reviewed by:  markj
Sponsored by:   The FreeBSD Foundation
MFC after:      1 week
Differential revision:  https://reviews.freebsd.org/D58463

libfetch: Fix handling of connection failures

After commit 848f360c8f9a, if one tries to connect to a closed port,
fetch reports "Operation now in progress", which is rather confusing.
Return a more useful error message, restoring the old behaviour.

Fixes:          https://cgit.freebsd.org/src/commit/?id=848f360c8f9a ("libfetch: Apply timeout to connection attempts")
Reviewed by:    des
MFC after:      3 days
Differential Revision:  https://reviews.freebsd.org/D58481

libfetch: Further improve connection polling

* Reorganize the connection loop to make it a little more readable

* Start the timeout clock earlier

* Correctly calculate the poll timeout before calling poll()

* Don't leak the socket on failure

Fixes:          https://cgit.freebsd.org/src/commit/?id=848f360c8f9a ("libfetch: Apply timeout to connection attempts")
Fixes:          https://cgit.freebsd.org/src/commit/?id=b02e02958dad ("libfetch: Fix handling of connection failures")
MFC after:      3 days
Reviewed by:    markj
Differential Revision:  https://reviews.freebsd.org/D58512

Some NLM_F_ definitions contain multiple underscores in their name; this
should pick them up.

Reviewed by:    kp, Ishan Agrawal <iagrawal9990@gmail.com>
Fixes:  https://cgit.freebsd.org/src/commit/?id=4c932a4d45fb ("netlink: decode netlink message flags symbolically")
Sponsored by:   The FreeBSD Foundation
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2340

Reviewed by:  jrtc27
Differential Revision:  https://reviews.freebsd.org/D58558

stdio: *memstream: slightly streamline growth function

Inverting the condition after realloc*() is a minor cleanup, but makes
the success path a little cleaner to ease a future change.

Reviewed by:    des, jhb
Sponsored by:   Klara, Inc.
Differential Revision:  https://reviews.freebsd.org/D57353

stdio: *memstream: decouple the buffer size from the stream length

It's useful to be able to track both facts with a single variable, but
it also makes it more difficult to change how the buffer size scales.

As an example, Apple's implementation seems to scale the buffer size by
1.5x on growth, presumably in an attempt to reduce trips into realloc().
This might be questionable in the face of stdio buffering, but avoiding
serious churn in the line- or un-buffered case is a net positive if
doing so isn't incredibly invasive.

Reviewed by:    des, jhb, obiwac
Sponsored by:   Klara, Inc.
Differential Revision:  https://reviews.freebsd.org/D57354

stdio: *memstream: grow the buffer by 1.5x on write

This improves performance by reducing the number of allocations as we
write into the memstream, both in the fully buffered case with larger
memstreams and also more trivially in the line- and un-buffered case as
they flush back to the underlying buffer more often.

The inspiration for this was taken from Apple's implementation in
https://github.com/apple-oss-distributions/libc, but expanded to include
wmemstream for consistency.  I've added a test for the bug that I hit in
libder that caused me to notice this in the first place, and fixed that
bug in this version.

Reviewed by:    des, jhb (both slightly previous version)
Sponsored by:   Klara, Inc.
Differential Revision:  https://reviews.freebsd.org/D57355

stringf.cc uses errno and related macros without including <cerrno>.

Their availability is guaranteed only when the corresponding header
is included; transitive exposure is implementation-defined.

Modern libc++ has been progressively reducing incidental transitive
includes as part of its header removal policy (see LLVM libc++ Header
Removal Policy and D132284), making such dependencies brittle.

This change includes <cerrno> explicitly to make the dependency
well-defined. No functional or behavioural change intended.

Approved by:    fuz
Signed-off-by:  Faraz Vahedi <kfv@kfv.io>
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2188

Currently mergesort() uses ICOPY_*() to copy data as four byte blocks
instead of one byte. However, this is only achievable when both size and
base arguments are aligned to four bytes.

Use of memcpy() is ideal as 1) it is cleaner and 2) the library will use
SIMD for copying when the hardware supports it. Compared to ICOPY_*(),
SIMD can support up to 64 bytes. When the SIMD-backed memcpy() find the
address is unaligned, it can first copy data up to the nearest aligned
address, and then use SIMD operations for faster transfer. Thus memcpy()
can give better performance than mergesort()'s own implementation.

This is benchmarked on amd64 where there isn't a SIMD-backed
implementation yet. However, the baseline implementation in assembly
already delivers better performance in unaligned cases although there is
some performance drops in aligned cases. The benchmark results and
script is available in the Phabricator review. Ideally, more performance
improvements will come when amd64 gets SIMD implementation of memcpy().

Signed-off-by:  Minsoo Choo <minsoochoo0122@proton.me>
Reviewed by:    fuz
MFC after:      1 week
Differential Revision:  https://reviews.freebsd.org/D58002

This commit implements the inverse half-cycle
trigonometric functions:

   asinpi(x) = asin(x) / pi                        Eq. (1)
   acospi(x) = acos(x) / pi
   atanpi(x) = atan(x) / pi

Implemention details are contained in src/s_asinpi.c and
src/a_atanpi.c, where the details for acospi(x) appear in
the former.

*************

CAVEAT EMPTOR: The ld128 code has been only compiled.  It has
not been tested for correctness due to lack of hardware.

*************

Code compiled on AMD Ryzen 7 7700X system run FreeBSD 16.0-CURRENT
main-n284956-de9fe28ab847.

Exhaustive testing of acospif(x), asinpif(x), and atanpif(x)
on the indicated intervals yields

% ./tlibm acospi -fPE -x 0x1p-120 -X 1
Interval tested for acospif: [7.52316e-37,1]
      ulp <= 0.5: 99.627% 1002878299 |  99.627% 1002878299
0.5 < ulp <= 0.6:  0.277%    2789599 |  99.904% 1005667898
0.6 < ulp <= 0.7:  0.096%     965062 | 100.000% 1006632960
Max ulp: 0.63661975 at 5.96046412e-08 0x1.fffffep-25

% ./tlibm asinpi -fPED -x 0x1p-120f -X 1.f
Interval tested for asinpif: [7.52316e-37,1]
      ulp <= 0.5: 99.851% 1005129353 |  99.851% 1005129353
0.5 < ulp <= 0.6:  0.149%    1501097 | 100.000% 1006630450
0.6 < ulp <= 0.7:  0.000%       2510 | 100.000% 1006632960
Max ulp: 0.68957579 at 5.04878759e-01 0x1.027f78p-1

% ./tlibm atanpi -fPE -x 0x1p-120 -X max > zatanpif.txt &
Interval tested for atanpif: [7.52316e-37,3.40282e+38]
      ulp <= 0.5: 99.865% 2077574602 |  99.865% 2077574602
0.5 < ulp <= 0.6:  0.131%    2735011 |  99.997% 2080309613
0.6 < ulp <= 0.7:  0.003%      65170 | 100.000% 2080374783
Max ulp: 0.68433094 at 5.01186252e-01 0x1.009b7cp-1

Testing the double and long double version cannot be done
in an exhaustive manner.  For 300 M values, uniformily
distributed in the indicated interals, one finds the max ULP:

Interval tested for acospi: [9.31323e-10,0.25]
  xm =  2.4423788416892520e-01, /* 0x3fcf432f, 0xde79920f */
libm =  4.2146222480005391e-01, /* 0x3fdaf93c, 0xb201001c */
mpfr =  4.2146222480005396e-01, /* 0x3fdaf93c, 0xb201001d */
 ULP = 0.50499351466286857

Interval tested for acospi: [0.25,0.5]
  xm =  4.9689430915631438e-01, /* 0x3fdfcd1d, 0xc9d945c6 */
libm =  3.3447366122373884e-01, /* 0x3fd56804, 0x371513ef */
mpfr =  3.3447366122373889e-01, /* 0x3fd56804, 0x371513f0 */
 ULP = 0.57195275455053829

Interval tested for acospi: [0.5,0.75]
  xm =  5.0238623667462079e-01, /* 0x3fe0138c, 0x4d0f4be0 */
libm =  3.3245556599062825e-01, /* 0x3fd546f3, 0xb5d36303 */
mpfr =  3.3245556599062820e-01, /* 0x3fd546f3, 0xb5d36302 */
 ULP = 0.63427929243758807

Interval tested for acospi: [0.75,1]
  xm =  7.5853651919512177e-01, /* 0x3fe845ee, 0x60d8789f */
libm =  2.2591472240382732e-01, /* 0x3fcceac6, 0x0c3465ce */
mpfr =  2.2591472240382729e-01, /* 0x3fcceac6, 0x0c3465cd */
 ULP = 0.56915750216472161

Interval tested for asinpi: [9.31323e-10,0.25]
  xm =  1.9502362835488171e-01, /* 0x3fc8f688, 0xc4dda0fb */
libm =  6.2478354989018887e-02, /* 0x3faffd29, 0xb6c57c61 */
mpfr =  6.2478354989018881e-02, /* 0x3faffd29, 0xb6c57c60 */
 ULP = 0.52347765415885006

Interval tested for asinpi: [0.25,0.5]
  xm =  4.9937103583123676e-01, /* 0x3fdff5b1, 0xeeddbf62 */
libm =  1.6643553767987129e-01, /* 0x3fc54dc2, 0x7b9d15a4 */
mpfr =  1.6643553767987126e-01, /* 0x3fc54dc2, 0x7b9d15a3 */
 ULP = 0.66214688371031072

Interval tested for asinpi: [0.5,0.75]
  xm =  5.0228515250761718e-01, /* 0x3fe012b8, 0x4fe92bbb */
libm =  1.6750722213679006e-01, /* 0x3fc570e0, 0x6c75edd5 */
mpfr =  1.6750722213679009e-01, /* 0x3fc570e0, 0x6c75edd6 */
 ULP = 0.78223048105528226

Interval tested for asinpi: [0.75,1]
  xm =  7.5425933001419776e-01, /* 0x3fe822e4, 0x7663a4aa */
libm =  2.7200385380185182e-01, /* 0x3fd16882, 0xda1dc13b */
mpfr =  2.7200385380185188e-01, /* 0x3fd16882, 0xda1dc13c */
 ULP = 0.53747973176773822

Interval tested for atanpi: [9.31323e-10,0.25]
  xm =  1.9666113418757322e-01, /* 0x3fc92c31, 0x29dd6d2f */
libm =  6.1810387818117797e-02, /* 0x3fafa59c, 0x7476baa5 */
mpfr =  6.1810387818117804e-02, /* 0x3fafa59c, 0x7476baa6 */
 ULP = 0.54674297446584263

Interval tested for atanpi: [0.25,0.5]
  xm =  4.1312119637707068e-01, /* 0x3fda7093, 0xe2ee5494 */
libm =  1.2470309560460152e-01, /* 0x3fbfec8a, 0xc554ebec */
mpfr =  1.2470309560460154e-01, /* 0x3fbfec8a, 0xc554ebed */
 ULP = 0.73116638175113347

Interval tested for atanpi: [0.5,0.75]
  xm =  5.0018949583396499e-01, /* 0x3fe0018d, 0x66cd1b82 */
libm =  1.4763186871058706e-01, /* 0x3fc2e599, 0xdffacb8f */
mpfr =  1.4763186871058709e-01, /* 0x3fc2e599, 0xdffacb90 */
 ULP = 0.69192753950764663

Interval tested for atanpi: [0.75,1]
  xm =  7.5007880583359599e-01, /* 0x3fe800a5, 0x448f4c03 */
libm =  2.0484881828445453e-01, /* 0x3fca387c, 0x6f93f71f */
mpfr =  2.0484881828445450e-01, /* 0x3fca387c, 0x6f93f71e */
 ULP = 0.65765471872064396

Interval tested for atanpi: [1,2]
  xm =  1.0103228000344093e+00, /* 0x3ff02a48, 0x3d88d0a2 */
libm =  2.5163447403817019e-01, /* 0x3fd01ac7, 0x7b229108 */
mpfr =  2.5163447403817013e-01, /* 0x3fd01ac7, 0x7b229107 */
 ULP = 0.67409519689166042

Interval tested for atanpi: [2,4]
  xm =  2.0231383267437946e+00, /* 0x40002f63, 0x25a530a9 */
libm =  3.5387589538123299e-01, /* 0x3fd6a5e7, 0x156053c6 */
mpfr =  3.5387589538123293e-01, /* 0x3fd6a5e7, 0x156053c5 */
 ULP = 0.69695587476021503

Interval tested for atanpi: [4,1.79769e+308]
  xm =  4.0000000000000000e+00, /* 0x40100000, 0x00000000 */
libm =  4.2202086962263069e-01, /* 0x3fdb0263, 0xd2508e31 */
mpfr =  4.2202086962263069e-01, /* 0x3fdb0263, 0xd2508e31 */
 ULP = 0.27709400511686716

PR:             https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=295884
MFC after:      1 month
Reviewed by:    fuz

strfromd(), strfromf(), and strfroml() are implemented directly
in terms of gdtoa.  If a non-conforming format string is passed,
the string "EDOOFUS" is returned and errno set to EDOOFUS as an
extension.

Reviewed by:    fuz
MFC after:      1 month
Pull-Request:   https://github.com/freebsd/freebsd-src/pull/2301
Signed-off-by:  Faraz Vahedi <kfv@kfv.io>

On some platforms, e.g. Linux Clang 22.1.8 / glibc 2.43, strchr()
now implements the C23 behaviour where passing a const pointer to
strchr() also returns a const pointer.  This breaks getopt during
the bootstrap build, since it assumes the return value is always
a mutable pointer.

Since the pointed-to value is never modified, fix this by making
the pointer const.

MFC after:      1 week
Reviewed by:    emaste
Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D58488

Add an uncached sysctl based implementation which retrieves individual
categories.  A cache would be an obvious extension should this optional
feature that can only be enabled by an environmental varible have a
noticable performance impact in a case that matters.

Reviewed by:    kib
Sponsored by:   Innovate UK
Differential Revision:  https://reviews.freebsd.org/D58238

When fts_children() is called, sp->fts_child is set. On the
next fts_read() call, fts_safe_changedir() was incorrectly
passed p->fts_dirfd (pointing to the parent directory) instead
of -1. This caused fts to fchdir to the parent instead of the
child directory, silently skipping the contents of 3rd-level
subdirectories. This was observed as a failure in
nmtree_test:mtree_create which calls fts_children() internally.

Add regression test accpath_correct_after_descent that calls
fts_children() on each directory entry and verifies files at
depth 3 are still visited correctly. The test fails with the
unfixed libc and passes with the fix.

Reported by:    Herbert J. Skuhra <herbert@gojira.at>
Sponsored by:   Google LLC (GSoC 2026)
Reviewed by:    asomers
Fixes:          https://cgit.freebsd.org/src/commit/?id=4bd01d6ae01 ("fts: refactor to use fd-relative")
Pull Request:   https://github.com/freebsd/freebsd-src/pull/2354