Print this page
10703 smatch unreachable code checking needs reworking
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
| Split |
Close |
| Expand all |
| Collapse all |
--- old/usr/src/uts/common/io/i40e/i40e_main.c
+++ new/usr/src/uts/common/io/i40e/i40e_main.c
1 1 /*
2 2 * This file and its contents are supplied under the terms of the
3 3 * Common Development and Distribution License ("CDDL"), version 1.0.
4 4 * You may only use this file in accordance with the terms of version
5 5 * 1.0 of the CDDL.
6 6 *
7 7 * A full copy of the text of the CDDL should have accompanied this
8 8 * source. A copy of the CDDL is also available via the Internet at
9 9 * http://www.illumos.org/license/CDDL.
10 10 */
11 11
12 12 /*
13 13 * Copyright 2015 OmniTI Computer Consulting, Inc. All rights reserved.
14 14 * Copyright 2019 Joyent, Inc.
15 15 * Copyright 2017 Tegile Systems, Inc. All rights reserved.
16 16 */
17 17
18 18 /*
19 19 * i40e - Intel 10/40 Gb Ethernet driver
20 20 *
21 21 * The i40e driver is the main software device driver for the Intel 40 Gb family
22 22 * of devices. Note that these devices come in many flavors with both 40 GbE
23 23 * ports and 10 GbE ports. This device is the successor to the 82599 family of
24 24 * devices (ixgbe).
25 25 *
26 26 * Unlike previous generations of Intel 1 GbE and 10 GbE devices, the 40 GbE
27 27 * devices defined in the XL710 controller (previously known as Fortville) are a
28 28 * rather different beast and have a small switch embedded inside of them. In
29 29 * addition, the way that most of the programming is done has been overhauled.
30 30 * As opposed to just using PCIe memory mapped registers, it also has an
31 31 * administrative queue which is used to communicate with firmware running on
32 32 * the chip.
33 33 *
34 34 * Each physical function in the hardware shows up as a device that this driver
35 35 * will bind to. The hardware splits many resources evenly across all of the
36 36 * physical functions present on the device, while other resources are instead
37 37 * shared across the entire card and its up to the device driver to
38 38 * intelligently partition them.
39 39 *
40 40 * ------------
41 41 * Organization
42 42 * ------------
43 43 *
44 44 * This driver is made up of several files which have their own theory
45 45 * statements spread across them. We'll touch on the high level purpose of each
46 46 * file here, and then we'll get into more discussion on how the device is
47 47 * generally modelled with respect to the interfaces in illumos.
48 48 *
49 49 * i40e_gld.c: This file contains all of the bindings to MAC and the networking
50 50 * stack.
51 51 *
52 52 * i40e_intr.c: This file contains all of the interrupt service routines and
53 53 * contains logic to enable and disable interrupts on the hardware.
54 54 * It also contains the logic to map hardware resources such as the
55 55 * rings to and from interrupts and controls their ability to fire.
56 56 *
57 57 * There is a big theory statement on interrupts present there.
58 58 *
59 59 * i40e_main.c: The file that you're currently in. It interfaces with the
60 60 * traditional OS DDI interfaces and is in charge of configuring
61 61 * the device.
62 62 *
63 63 * i40e_osdep.[ch]: These files contain interfaces and definitions needed to
64 64 * work with Intel's common code for the device.
65 65 *
66 66 * i40e_stats.c: This file contains the general work and logic around our
67 67 * kstats. A theory statement on their organization and use of the
68 68 * hardware exists there.
69 69 *
70 70 * i40e_sw.h: This header file contains all of the primary structure definitions
71 71 * and constants that are used across the entire driver.
72 72 *
73 73 * i40e_transceiver.c: This file contains all of the logic for sending and
74 74 * receiving data. It contains all of the ring and DMA
75 75 * allocation logic, as well as, the actual interfaces to
76 76 * send and receive data.
77 77 *
78 78 * A big theory statement on ring management, descriptors,
79 79 * and how it ties into the OS is present there.
80 80 *
81 81 * --------------
82 82 * General Design
83 83 * --------------
84 84 *
85 85 * Before we go too far into the general way we've laid out data structures and
86 86 * the like, it's worth taking some time to explain how the hardware is
87 87 * organized. This organization informs a lot of how we do things at this time
88 88 * in the driver.
89 89 *
90 90 * Each physical device consists of a number of one or more ports, which are
91 91 * considered physical functions in the PCI sense and thus each get enumerated
92 92 * by the system, resulting in an instance being created and attached to. While
93 93 * there are many resources that are unique to each physical function eg.
94 94 * instance of the device, there are many that are shared across all of them.
95 95 * Several resources have an amount reserved for each Virtual Station Interface
96 96 * (VSI) and then a static pool of resources, available for all functions on the
97 97 * card.
98 98 *
99 99 * The most important resource in hardware are its transmit and receive queue
100 100 * pairs (i40e_trqpair_t). These should be thought of as rings in GLDv3
101 101 * parlance. There are a set number of these on each device; however, they are
102 102 * statically partitioned among all of the different physical functions.
103 103 *
104 104 * 'Fortville' (the code name for this device family) is basically a switch. To
105 105 * map MAC addresses and other things to queues, we end up having to create
106 106 * Virtual Station Interfaces (VSIs) and establish forwarding rules that direct
107 107 * traffic to a queue. A VSI owns a collection of queues and has a series of
108 108 * forwarding rules that point to it. One way to think of this is to treat it
109 109 * like MAC does a VNIC. When MAC refers to a group, a collection of rings and
110 110 * classification resources, that is a VSI in i40e.
111 111 *
112 112 * The sets of VSIs is shared across the entire device, though there may be some
113 113 * amount that are reserved to each PF. Because the GLDv3 does not let us change
114 114 * the number of groups dynamically, we instead statically divide this amount
115 115 * evenly between all the functions that exist. In addition, we have the same
116 116 * problem with the mac address forwarding rules. There are a static number that
117 117 * exist shared across all the functions.
118 118 *
119 119 * To handle both of these resources, what we end up doing is going through and
120 120 * determining which functions belong to the same device. Nominally one might do
121 121 * this by having a nexus driver; however, a prime requirement for a nexus
122 122 * driver is identifying the various children and activating them. While it is
123 123 * possible to get this information from NVRAM, we would end up duplicating a
124 124 * lot of the PCI enumeration logic. Really, at the end of the day, the device
125 125 * doesn't give us the traditional identification properties we want from a
126 126 * nexus driver.
127 127 *
128 128 * Instead, we rely on some properties that are guaranteed to be unique. While
129 129 * it might be tempting to leverage the PBA or serial number of the device from
130 130 * NVRAM, there is nothing that says that two devices can't be mis-programmed to
131 131 * have the same values in NVRAM. Instead, we uniquely identify a group of
132 132 * functions based on their parent in the /devices tree, their PCI bus and PCI
133 133 * function identifiers. Using either on their own may not be sufficient.
134 134 *
135 135 * For each unique PCI device that we encounter, we'll create a i40e_device_t.
136 136 * From there, because we don't have a good way to tell the GLDv3 about sharing
137 137 * resources between everything, we'll end up just dividing the resources
138 138 * evenly between all of the functions. Longer term, if we don't have to declare
139 139 * to the GLDv3 that these resources are shared, then we'll maintain a pool and
140 140 * have each PF allocate from the pool in the device, thus if only two of four
141 141 * ports are being used, for example, then all of the resources can still be
142 142 * used.
143 143 *
144 144 * -------------------------------------------
145 145 * Transmit and Receive Queue Pair Allocations
146 146 * -------------------------------------------
147 147 *
148 148 * NVRAM ends up assigning each PF its own share of the transmit and receive LAN
149 149 * queue pairs, we have no way of modifying it, only observing it. From there,
150 150 * it's up to us to map these queues to VSIs and VFs. Since we don't support any
151 151 * VFs at this time, we only focus on assignments to VSIs.
152 152 *
153 153 * At the moment, we used a static mapping of transmit/receive queue pairs to a
154 154 * given VSI (eg. rings to a group). Though in the fullness of time, we want to
155 155 * make this something which is fully dynamic and take advantage of documented,
156 156 * but not yet available functionality for adding filters based on VXLAN and
157 157 * other encapsulation technologies.
158 158 *
159 159 * -------------------------------------
160 160 * Broadcast, Multicast, and Promiscuous
161 161 * -------------------------------------
162 162 *
163 163 * As part of the GLDv3, we need to make sure that we can handle receiving
164 164 * broadcast and multicast traffic. As well as enabling promiscuous mode when
165 165 * requested. GLDv3 requires that all broadcast and multicast traffic be
166 166 * retrieved by the default group, eg. the first one. This is the same thing as
167 167 * the default VSI.
168 168 *
169 169 * To receieve broadcast traffic, we enable it through the admin queue, rather
170 170 * than use one of our filters for it. For multicast traffic, we reserve a
171 171 * certain number of the hash filters and assign them to a given PF. When we
172 172 * exceed those, we then switch to using promiscuous mode for multicast traffic.
173 173 *
174 174 * More specifically, once we exceed the number of filters (indicated because
175 175 * the i40e_t`i40e_resources.ifr_nmcastfilt ==
176 176 * i40e_t`i40e_resources.ifr_nmcastfilt_used), we then instead need to toggle
177 177 * promiscuous mode. If promiscuous mode is toggled then we keep track of the
178 178 * number of MACs added to it by incrementing i40e_t`i40e_mcast_promisc_count.
179 179 * That will stay enabled until that count reaches zero indicating that we have
180 180 * only added multicast addresses that we have a corresponding entry for.
181 181 *
182 182 * Because MAC itself wants to toggle promiscuous mode, which includes both
183 183 * unicast and multicast traffic, we go through and keep track of that
184 184 * ourselves. That is maintained through the use of the i40e_t`i40e_promisc_on
185 185 * member.
186 186 *
187 187 * --------------
188 188 * VSI Management
189 189 * --------------
190 190 *
191 191 * The PFs share 384 VSIs. The firmware creates one VSI per PF by default.
192 192 * During chip start we retrieve the SEID of this VSI and assign it as the
193 193 * default VSI for our VEB (one VEB per PF). We then add additional VSIs to
194 194 * the VEB up to the determined number of rx groups: i40e_t`i40e_num_rx_groups.
195 195 * We currently cap this number to I40E_GROUP_MAX to a) make sure all PFs can
196 196 * allocate the same number of VSIs, and b) to keep the interrupt multiplexing
197 197 * under control. In the future, when we improve the interrupt allocation, we
198 198 * may want to revisit this cap to make better use of the available VSIs. The
199 199 * VSI allocation and configuration can be found in i40e_chip_start().
200 200 *
201 201 * ----------------
202 202 * Structure Layout
203 203 * ----------------
204 204 *
205 205 * The following images relates the core data structures together. The primary
206 206 * structure in the system is the i40e_t. It itself contains multiple rings,
207 207 * i40e_trqpair_t's which contain the various transmit and receive data. The
208 208 * receive data is stored outside of the i40e_trqpair_t and instead in the
209 209 * i40e_rx_data_t. The i40e_t has a corresponding i40e_device_t which keeps
210 210 * track of per-physical device state. Finally, for every active descriptor,
211 211 * there is a corresponding control block, which is where the
212 212 * i40e_rx_control_block_t and the i40e_tx_control_block_t come from.
213 213 *
214 214 * +-----------------------+ +-----------------------+
215 215 * | Global i40e_t list | | Global Device list |
216 216 * | | +--| |
217 217 * | i40e_glist | | | i40e_dlist |
218 218 * +-----------------------+ | +-----------------------+
219 219 * | v
220 220 * | +------------------------+ +-----------------------+
221 221 * | | Device-wide Structure |----->| Device-wide Structure |--> ...
222 222 * | | i40e_device_t | | i40e_device_t |
223 223 * | | | +-----------------------+
224 224 * | | dev_info_t * ------+--> Parent in devices tree.
225 225 * | | uint_t ------+--> PCI bus number
226 226 * | | uint_t ------+--> PCI device number
227 227 * | | uint_t ------+--> Number of functions
228 228 * | | i40e_switch_rsrcs_t ---+--> Captured total switch resources
229 229 * | | list_t ------+-------------+
230 230 * | +------------------------+ |
231 231 * | ^ |
232 232 * | +--------+ |
233 233 * | | v
234 234 * | +---------------------------+ | +-------------------+
235 235 * +->| GLDv3 Device, per PF |-----|-->| GLDv3 Device (PF) |--> ...
236 236 * | i40e_t | | | i40e_t |
237 237 * | **Primary Structure** | | +-------------------+
238 238 * | | |
239 239 * | i40e_device_t * --+-----+
240 240 * | i40e_state_t --+---> Device State
241 241 * | i40e_hw_t --+---> Intel common code structure
242 242 * | mac_handle_t --+---> GLDv3 handle to MAC
243 243 * | ddi_periodic_t --+---> Link activity timer
244 244 * | i40e_vsi_t * --+---> Array of VSIs
245 245 * | i40e_func_rsrc_t --+---> Available hardware resources
246 246 * | i40e_switch_rsrc_t * --+---> Switch resource snapshot
247 247 * | i40e_sdu --+---> Current MTU
248 248 * | i40e_frame_max --+---> Current HW frame size
249 249 * | i40e_uaddr_t * --+---> Array of assigned unicast MACs
250 250 * | i40e_maddr_t * --+---> Array of assigned multicast MACs
251 251 * | i40e_mcast_promisccount --+---> Active multicast state
252 252 * | i40e_promisc_on --+---> Current promiscuous mode state
253 253 * | uint_t --+---> Number of transmit/receive pairs
254 254 * | i40e_rx_group_t * --+---> Array of Rx groups
255 255 * | kstat_t * --+---> PF kstats
256 256 * | i40e_pf_stats_t --+---> PF kstat backing data
257 257 * | i40e_trqpair_t * --+---------+
258 258 * +---------------------------+ |
259 259 * |
260 260 * v
261 261 * +-------------------------------+ +-----------------------------+
262 262 * | Transmit/Receive Queue Pair |-------| Transmit/Receive Queue Pair |->...
263 263 * | i40e_trqpair_t | | i40e_trqpair_t |
264 264 * + Ring Data Structure | +-----------------------------+
265 265 * | |
266 266 * | mac_ring_handle_t +--> MAC RX ring handle
267 267 * | mac_ring_handle_t +--> MAC TX ring handle
268 268 * | i40e_rxq_stat_t --+--> RX Queue stats
269 269 * | i40e_txq_stat_t --+--> TX Queue stats
270 270 * | uint32_t (tx ring size) +--> TX Ring Size
271 271 * | uint32_t (tx free list size) +--> TX Free List Size
272 272 * | i40e_dma_buffer_t --------+--> TX Descriptor ring DMA
273 273 * | i40e_tx_desc_t * --------+--> TX descriptor ring
274 274 * | volatile unt32_t * +--> TX Write back head
275 275 * | uint32_t -------+--> TX ring head
276 276 * | uint32_t -------+--> TX ring tail
277 277 * | uint32_t -------+--> Num TX desc free
278 278 * | i40e_tx_control_block_t * --+--> TX control block array ---+
279 279 * | i40e_tx_control_block_t ** --+--> TCB work list ----+
280 280 * | i40e_tx_control_block_t ** --+--> TCB free list ---+
281 281 * | uint32_t -------+--> Free TCB count |
282 282 * | i40e_rx_data_t * -------+--+ v
283 283 * +-------------------------------+ | +---------------------------+
284 284 * | | Per-TX Frame Metadata |
285 285 * | | i40e_tx_control_block_t |
286 286 * +--------------------+ | |
287 287 * | mblk to transmit <--+--- mblk_t * |
288 288 * | type of transmit <--+--- i40e_tx_type_t |
289 289 * | TX DMA handle <--+--- ddi_dma_handle_t |
290 290 * v TX DMA buffer <--+--- i40e_dma_buffer_t |
291 291 * +------------------------------+ +---------------------------+
292 292 * | Core Receive Data |
293 293 * | i40e_rx_data_t |
294 294 * | |
295 295 * | i40e_dma_buffer_t --+--> RX descriptor DMA Data
296 296 * | i40e_rx_desc_t --+--> RX descriptor ring
297 297 * | uint32_t --+--> Next free desc.
298 298 * | i40e_rx_control_block_t * --+--> RX Control Block Array ---+
299 299 * | i40e_rx_control_block_t ** --+--> RCB work list ---+
300 300 * | i40e_rx_control_block_t ** --+--> RCB free list ---+
301 301 * +------------------------------+ |
302 302 * ^ |
303 303 * | +---------------------------+ |
304 304 * | | Per-RX Frame Metadata |<---------------+
305 305 * | | i40e_rx_control_block_t |
306 306 * | | |
307 307 * | | mblk_t * ----+--> Received mblk_t data
308 308 * | | uint32_t ----+--> Reference count
309 309 * | | i40e_dma_buffer_t ----+--> Receive data DMA info
310 310 * | | frtn_t ----+--> mblk free function info
311 311 * +-----+-- i40e_rx_data_t * |
312 312 * +---------------------------+
313 313 *
314 314 * -------------
315 315 * Lock Ordering
316 316 * -------------
317 317 *
318 318 * In order to ensure that we don't deadlock, the following represents the
319 319 * lock order being used. When grabbing locks, follow the following order. Lower
320 320 * numbers are more important. Thus, the i40e_glock which is number 0, must be
321 321 * taken before any other locks in the driver. On the other hand, the
322 322 * i40e_t`i40e_stat_lock, has the highest number because it's the least
323 323 * important lock. Note, that just because one lock is higher than another does
324 324 * not mean that all intermediary locks are required.
325 325 *
326 326 * 0) i40e_glock
327 327 * 1) i40e_t`i40e_general_lock
328 328 *
329 329 * 2) i40e_trqpair_t`itrq_rx_lock
330 330 * 3) i40e_trqpair_t`itrq_tx_lock
331 331 * 4) i40e_t`i40e_rx_pending_lock
332 332 * 5) i40e_trqpair_t`itrq_tcb_lock
333 333 *
334 334 * 6) i40e_t`i40e_stat_lock
335 335 *
336 336 * Rules and expectations:
337 337 *
338 338 * 1) A thread holding locks belong to one PF should not hold locks belonging to
339 339 * a second. If for some reason this becomes necessary, locks should be grabbed
340 340 * based on the list order in the i40e_device_t, which implies that the
341 341 * i40e_glock is held.
342 342 *
343 343 * 2) When grabbing locks between multiple transmit and receive queues, the
344 344 * locks for the lowest number transmit/receive queue should be grabbed first.
345 345 *
346 346 * 3) When grabbing both the transmit and receive lock for a given queue, always
347 347 * grab i40e_trqpair_t`itrq_rx_lock before the i40e_trqpair_t`itrq_tx_lock.
348 348 *
349 349 * 4) The following pairs of locks are not expected to be held at the same time:
350 350 *
351 351 * o i40e_t`i40e_rx_pending_lock and i40e_trqpair_t`itrq_tcb_lock
352 352 *
353 353 * -----------
354 354 * Future Work
355 355 * -----------
356 356 *
357 357 * At the moment the i40e_t driver is rather bare bones, allowing us to start
358 358 * getting data flowing and folks using it while we develop additional features.
359 359 * While bugs have been filed to cover this future work, the following gives an
360 360 * overview of expected work:
361 361 *
362 362 * o DMA binding and breaking up the locking in ring recycling.
363 363 * o Enhanced detection of device errors
364 364 * o Participation in IRM
365 365 * o FMA device reset
366 366 * o Stall detection, temperature error detection, etc.
367 367 * o More dynamic resource pools
368 368 */
369 369
370 370 #include "i40e_sw.h"
371 371
372 372 static char i40e_ident[] = "Intel 10/40Gb Ethernet v1.0.3";
373 373
374 374 /*
375 375 * The i40e_glock primarily protects the lists below and the i40e_device_t
376 376 * structures.
377 377 */
378 378 static kmutex_t i40e_glock;
379 379 static list_t i40e_glist;
380 380 static list_t i40e_dlist;
381 381
382 382 /*
383 383 * Access attributes for register mapping.
384 384 */
385 385 static ddi_device_acc_attr_t i40e_regs_acc_attr = {
386 386 DDI_DEVICE_ATTR_V1,
387 387 DDI_STRUCTURE_LE_ACC,
388 388 DDI_STRICTORDER_ACC,
389 389 DDI_FLAGERR_ACC
390 390 };
391 391
392 392 /*
393 393 * Logging function for this driver.
394 394 */
395 395 static void
396 396 i40e_dev_err(i40e_t *i40e, int level, boolean_t console, const char *fmt,
397 397 va_list ap)
398 398 {
399 399 char buf[1024];
400 400
401 401 (void) vsnprintf(buf, sizeof (buf), fmt, ap);
402 402
403 403 if (i40e == NULL) {
404 404 cmn_err(level, (console) ? "%s: %s" : "!%s: %s",
405 405 I40E_MODULE_NAME, buf);
406 406 } else {
407 407 dev_err(i40e->i40e_dip, level, (console) ? "%s" : "!%s",
408 408 buf);
409 409 }
410 410 }
411 411
412 412 /*
413 413 * Because there's the stupid trailing-comma problem with the C preprocessor
414 414 * and variable arguments, I need to instantiate these. Pardon the redundant
415 415 * code.
416 416 */
417 417 /*PRINTFLIKE2*/
418 418 void
419 419 i40e_error(i40e_t *i40e, const char *fmt, ...)
420 420 {
421 421 va_list ap;
422 422
423 423 va_start(ap, fmt);
424 424 i40e_dev_err(i40e, CE_WARN, B_FALSE, fmt, ap);
425 425 va_end(ap);
426 426 }
427 427
428 428 /*PRINTFLIKE2*/
429 429 void
430 430 i40e_log(i40e_t *i40e, const char *fmt, ...)
431 431 {
432 432 va_list ap;
433 433
434 434 va_start(ap, fmt);
435 435 i40e_dev_err(i40e, CE_NOTE, B_FALSE, fmt, ap);
436 436 va_end(ap);
437 437 }
438 438
439 439 /*PRINTFLIKE2*/
440 440 void
441 441 i40e_notice(i40e_t *i40e, const char *fmt, ...)
442 442 {
443 443 va_list ap;
444 444
445 445 va_start(ap, fmt);
446 446 i40e_dev_err(i40e, CE_NOTE, B_TRUE, fmt, ap);
447 447 va_end(ap);
448 448 }
449 449
450 450 /*
451 451 * Various parts of the driver need to know if the controller is from the X722
452 452 * family, which has a few additional capabilities and different programming
453 453 * means. We don't consider virtual functions as part of this as they are quite
454 454 * different and will require substantially more work.
455 455 */
456 456 static boolean_t
457 457 i40e_is_x722(i40e_t *i40e)
458 458 {
459 459 return (i40e->i40e_hw_space.mac.type == I40E_MAC_X722);
460 460 }
461 461
462 462 static void
463 463 i40e_device_rele(i40e_t *i40e)
464 464 {
465 465 i40e_device_t *idp = i40e->i40e_device;
466 466
467 467 if (idp == NULL)
468 468 return;
469 469
470 470 mutex_enter(&i40e_glock);
471 471 VERIFY(idp->id_nreg > 0);
472 472 list_remove(&idp->id_i40e_list, i40e);
473 473 idp->id_nreg--;
474 474 if (idp->id_nreg == 0) {
475 475 list_remove(&i40e_dlist, idp);
476 476 list_destroy(&idp->id_i40e_list);
477 477 kmem_free(idp->id_rsrcs, sizeof (i40e_switch_rsrc_t) *
478 478 idp->id_rsrcs_alloc);
479 479 kmem_free(idp, sizeof (i40e_device_t));
480 480 }
481 481 i40e->i40e_device = NULL;
482 482 mutex_exit(&i40e_glock);
483 483 }
484 484
485 485 static i40e_device_t *
486 486 i40e_device_find(i40e_t *i40e, dev_info_t *parent, uint_t bus, uint_t device)
487 487 {
488 488 i40e_device_t *idp;
489 489 mutex_enter(&i40e_glock);
490 490 for (idp = list_head(&i40e_dlist); idp != NULL;
491 491 idp = list_next(&i40e_dlist, idp)) {
492 492 if (idp->id_parent == parent && idp->id_pci_bus == bus &&
493 493 idp->id_pci_device == device) {
494 494 break;
495 495 }
496 496 }
497 497
498 498 if (idp != NULL) {
499 499 VERIFY(idp->id_nreg < idp->id_nfuncs);
500 500 idp->id_nreg++;
501 501 } else {
502 502 i40e_hw_t *hw = &i40e->i40e_hw_space;
503 503 ASSERT(hw->num_ports > 0);
504 504 ASSERT(hw->num_partitions > 0);
505 505
506 506 /*
507 507 * The Intel common code doesn't exactly keep the number of PCI
508 508 * functions. But it calculates it during discovery of
509 509 * partitions and ports. So what we do is undo the calculation
510 510 * that it does originally, as functions are evenly spread
511 511 * across ports in the rare case of partitions.
512 512 */
513 513 idp = kmem_alloc(sizeof (i40e_device_t), KM_SLEEP);
514 514 idp->id_parent = parent;
515 515 idp->id_pci_bus = bus;
516 516 idp->id_pci_device = device;
517 517 idp->id_nfuncs = hw->num_ports * hw->num_partitions;
518 518 idp->id_nreg = 1;
519 519 idp->id_rsrcs_alloc = i40e->i40e_switch_rsrc_alloc;
520 520 idp->id_rsrcs_act = i40e->i40e_switch_rsrc_actual;
521 521 idp->id_rsrcs = kmem_alloc(sizeof (i40e_switch_rsrc_t) *
522 522 idp->id_rsrcs_alloc, KM_SLEEP);
523 523 bcopy(i40e->i40e_switch_rsrcs, idp->id_rsrcs,
524 524 sizeof (i40e_switch_rsrc_t) * idp->id_rsrcs_alloc);
525 525 list_create(&idp->id_i40e_list, sizeof (i40e_t),
526 526 offsetof(i40e_t, i40e_dlink));
527 527
528 528 list_insert_tail(&i40e_dlist, idp);
529 529 }
530 530
531 531 list_insert_tail(&idp->id_i40e_list, i40e);
532 532 mutex_exit(&i40e_glock);
533 533
534 534 return (idp);
535 535 }
536 536
537 537 static void
538 538 i40e_link_state_set(i40e_t *i40e, link_state_t state)
539 539 {
540 540 if (i40e->i40e_link_state == state)
541 541 return;
542 542
543 543 i40e->i40e_link_state = state;
544 544 mac_link_update(i40e->i40e_mac_hdl, i40e->i40e_link_state);
545 545 }
546 546
547 547 /*
548 548 * This is a basic link check routine. Mostly we're using this just to see
549 549 * if we can get any accurate information about the state of the link being
550 550 * up or down, as well as updating the link state, speed, etc. information.
551 551 */
552 552 void
553 553 i40e_link_check(i40e_t *i40e)
554 554 {
555 555 i40e_hw_t *hw = &i40e->i40e_hw_space;
556 556 boolean_t ls;
557 557 int ret;
558 558
559 559 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
560 560
561 561 hw->phy.get_link_info = B_TRUE;
562 562 if ((ret = i40e_get_link_status(hw, &ls)) != I40E_SUCCESS) {
563 563 i40e->i40e_s_link_status_errs++;
564 564 i40e->i40e_s_link_status_lasterr = ret;
565 565 return;
566 566 }
567 567
568 568 /*
569 569 * Firmware abstracts all of the mac and phy information for us, so we
570 570 * can use i40e_get_link_status to determine the current state.
571 571 */
572 572 if (ls == B_TRUE) {
573 573 enum i40e_aq_link_speed speed;
574 574
575 575 speed = i40e_get_link_speed(hw);
576 576
577 577 /*
578 578 * Translate from an i40e value to a value in Mbits/s.
579 579 */
580 580 switch (speed) {
581 581 case I40E_LINK_SPEED_100MB:
582 582 i40e->i40e_link_speed = 100;
583 583 break;
584 584 case I40E_LINK_SPEED_1GB:
585 585 i40e->i40e_link_speed = 1000;
586 586 break;
587 587 case I40E_LINK_SPEED_10GB:
588 588 i40e->i40e_link_speed = 10000;
589 589 break;
590 590 case I40E_LINK_SPEED_20GB:
591 591 i40e->i40e_link_speed = 20000;
592 592 break;
593 593 case I40E_LINK_SPEED_40GB:
594 594 i40e->i40e_link_speed = 40000;
595 595 break;
596 596 case I40E_LINK_SPEED_25GB:
597 597 i40e->i40e_link_speed = 25000;
598 598 break;
599 599 default:
600 600 i40e->i40e_link_speed = 0;
601 601 break;
602 602 }
603 603
604 604 /*
605 605 * At this time, hardware does not support half-duplex
606 606 * operation, hence why we don't ask the hardware about our
607 607 * current speed.
608 608 */
609 609 i40e->i40e_link_duplex = LINK_DUPLEX_FULL;
610 610 i40e_link_state_set(i40e, LINK_STATE_UP);
611 611 } else {
612 612 i40e->i40e_link_speed = 0;
613 613 i40e->i40e_link_duplex = 0;
614 614 i40e_link_state_set(i40e, LINK_STATE_DOWN);
615 615 }
616 616 }
617 617
618 618 static void
619 619 i40e_rem_intrs(i40e_t *i40e)
620 620 {
621 621 int i, rc;
622 622
623 623 for (i = 0; i < i40e->i40e_intr_count; i++) {
624 624 rc = ddi_intr_free(i40e->i40e_intr_handles[i]);
625 625 if (rc != DDI_SUCCESS) {
626 626 i40e_log(i40e, "failed to free interrupt %d: %d",
627 627 i, rc);
628 628 }
629 629 }
630 630
631 631 kmem_free(i40e->i40e_intr_handles, i40e->i40e_intr_size);
632 632 i40e->i40e_intr_handles = NULL;
633 633 }
634 634
635 635 static void
636 636 i40e_rem_intr_handlers(i40e_t *i40e)
637 637 {
638 638 int i, rc;
639 639
640 640 for (i = 0; i < i40e->i40e_intr_count; i++) {
641 641 rc = ddi_intr_remove_handler(i40e->i40e_intr_handles[i]);
642 642 if (rc != DDI_SUCCESS) {
643 643 i40e_log(i40e, "failed to remove interrupt %d: %d",
644 644 i, rc);
645 645 }
646 646 }
647 647 }
648 648
649 649 /*
650 650 * illumos Fault Management Architecture (FMA) support.
651 651 */
652 652
653 653 int
654 654 i40e_check_acc_handle(ddi_acc_handle_t handle)
655 655 {
656 656 ddi_fm_error_t de;
657 657
658 658 ddi_fm_acc_err_get(handle, &de, DDI_FME_VERSION);
659 659 ddi_fm_acc_err_clear(handle, DDI_FME_VERSION);
660 660 return (de.fme_status);
661 661 }
662 662
663 663 int
664 664 i40e_check_dma_handle(ddi_dma_handle_t handle)
665 665 {
666 666 ddi_fm_error_t de;
667 667
668 668 ddi_fm_dma_err_get(handle, &de, DDI_FME_VERSION);
669 669 return (de.fme_status);
670 670 }
671 671
672 672 /*
673 673 * Fault service error handling callback function.
674 674 */
675 675 /* ARGSUSED */
676 676 static int
677 677 i40e_fm_error_cb(dev_info_t *dip, ddi_fm_error_t *err, const void *impl_data)
678 678 {
679 679 pci_ereport_post(dip, err, NULL);
680 680 return (err->fme_status);
681 681 }
682 682
683 683 static void
684 684 i40e_fm_init(i40e_t *i40e)
685 685 {
686 686 ddi_iblock_cookie_t iblk;
687 687
688 688 i40e->i40e_fm_capabilities = ddi_prop_get_int(DDI_DEV_T_ANY,
689 689 i40e->i40e_dip, DDI_PROP_DONTPASS, "fm_capable",
690 690 DDI_FM_EREPORT_CAPABLE | DDI_FM_ACCCHK_CAPABLE |
691 691 DDI_FM_DMACHK_CAPABLE | DDI_FM_ERRCB_CAPABLE);
692 692
693 693 if (i40e->i40e_fm_capabilities < 0) {
694 694 i40e->i40e_fm_capabilities = 0;
695 695 } else if (i40e->i40e_fm_capabilities > 0xf) {
696 696 i40e->i40e_fm_capabilities = DDI_FM_EREPORT_CAPABLE |
697 697 DDI_FM_ACCCHK_CAPABLE | DDI_FM_DMACHK_CAPABLE |
698 698 DDI_FM_ERRCB_CAPABLE;
699 699 }
700 700
701 701 /*
702 702 * Only register with IO Fault Services if we have some capability
703 703 */
704 704 if (i40e->i40e_fm_capabilities & DDI_FM_ACCCHK_CAPABLE) {
705 705 i40e_regs_acc_attr.devacc_attr_access = DDI_FLAGERR_ACC;
706 706 } else {
707 707 i40e_regs_acc_attr.devacc_attr_access = DDI_DEFAULT_ACC;
708 708 }
709 709
710 710 if (i40e->i40e_fm_capabilities) {
711 711 ddi_fm_init(i40e->i40e_dip, &i40e->i40e_fm_capabilities, &iblk);
712 712
713 713 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) ||
714 714 DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) {
715 715 pci_ereport_setup(i40e->i40e_dip);
716 716 }
717 717
718 718 if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) {
719 719 ddi_fm_handler_register(i40e->i40e_dip,
720 720 i40e_fm_error_cb, (void*)i40e);
721 721 }
722 722 }
723 723
724 724 if (i40e->i40e_fm_capabilities & DDI_FM_DMACHK_CAPABLE) {
725 725 i40e_init_dma_attrs(i40e, B_TRUE);
726 726 } else {
727 727 i40e_init_dma_attrs(i40e, B_FALSE);
728 728 }
729 729 }
730 730
731 731 static void
732 732 i40e_fm_fini(i40e_t *i40e)
733 733 {
734 734 if (i40e->i40e_fm_capabilities) {
735 735
736 736 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) ||
737 737 DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities))
738 738 pci_ereport_teardown(i40e->i40e_dip);
739 739
740 740 if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities))
741 741 ddi_fm_handler_unregister(i40e->i40e_dip);
742 742
743 743 ddi_fm_fini(i40e->i40e_dip);
744 744 }
745 745 }
746 746
747 747 void
748 748 i40e_fm_ereport(i40e_t *i40e, char *detail)
749 749 {
750 750 uint64_t ena;
751 751 char buf[FM_MAX_CLASS];
752 752
753 753 (void) snprintf(buf, FM_MAX_CLASS, "%s.%s", DDI_FM_DEVICE, detail);
754 754 ena = fm_ena_generate(0, FM_ENA_FMT1);
755 755 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities)) {
756 756 ddi_fm_ereport_post(i40e->i40e_dip, buf, ena, DDI_NOSLEEP,
757 757 FM_VERSION, DATA_TYPE_UINT8, FM_EREPORT_VERS0, NULL);
758 758 }
759 759 }
760 760
761 761 /*
762 762 * Here we're trying to set the SEID of the default VSI. In general,
763 763 * when we come through and look at this shortly after attach, we
764 764 * expect there to only be a single element present, which is the
765 765 * default VSI. Importantly, each PF seems to not see any other
766 766 * devices, in part because of the simple switch mode that we're
767 767 * using. If for some reason, we see more artifacts, we'll need to
768 768 * revisit what we're doing here.
769 769 */
770 770 static boolean_t
771 771 i40e_set_def_vsi_seid(i40e_t *i40e)
772 772 {
773 773 i40e_hw_t *hw = &i40e->i40e_hw_space;
774 774 struct i40e_aqc_get_switch_config_resp *sw_config;
775 775 uint8_t aq_buf[I40E_AQ_LARGE_BUF];
776 776 uint16_t next = 0;
777 777 int rc;
778 778
779 779 /* LINTED: E_BAD_PTR_CAST_ALIGN */
780 780 sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf;
781 781 rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next,
782 782 NULL);
783 783 if (rc != I40E_SUCCESS) {
784 784 i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d",
785 785 rc, hw->aq.asq_last_status);
786 786 return (B_FALSE);
787 787 }
788 788
789 789 if (LE_16(sw_config->header.num_reported) != 1) {
790 790 i40e_error(i40e, "encountered multiple (%d) switching units "
791 791 "during attach, not proceeding",
792 792 LE_16(sw_config->header.num_reported));
793 793 return (B_FALSE);
794 794 }
795 795
796 796 I40E_DEF_VSI_SEID(i40e) = sw_config->element[0].seid;
797 797 return (B_TRUE);
798 798 }
799 799
800 800 /*
801 801 * Get the SEID of the uplink MAC.
802 802 */
803 803 static int
804 804 i40e_get_mac_seid(i40e_t *i40e)
805 805 {
806 806 i40e_hw_t *hw = &i40e->i40e_hw_space;
807 807 struct i40e_aqc_get_switch_config_resp *sw_config;
808 808 uint8_t aq_buf[I40E_AQ_LARGE_BUF];
809 809 uint16_t next = 0;
810 810 int rc;
811 811
812 812 /* LINTED: E_BAD_PTR_CAST_ALIGN */
813 813 sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf;
814 814 rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next,
815 815 NULL);
816 816 if (rc != I40E_SUCCESS) {
817 817 i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d",
818 818 rc, hw->aq.asq_last_status);
819 819 return (-1);
820 820 }
821 821
822 822 return (LE_16(sw_config->element[0].uplink_seid));
823 823 }
824 824
825 825 /*
826 826 * We need to fill the i40e_hw_t structure with the capabilities of this PF. We
827 827 * must also provide the memory for it; however, we don't need to keep it around
828 828 * to the call to the common code. It takes it and parses it into an internal
829 829 * structure.
830 830 */
831 831 static boolean_t
832 832 i40e_get_hw_capabilities(i40e_t *i40e, i40e_hw_t *hw)
833 833 {
834 834 struct i40e_aqc_list_capabilities_element_resp *buf;
835 835 int rc;
836 836 size_t len;
837 837 uint16_t needed;
838 838 int nelems = I40E_HW_CAP_DEFAULT;
839 839
840 840 len = nelems * sizeof (*buf);
841 841
842 842 for (;;) {
843 843 ASSERT(len > 0);
844 844 buf = kmem_alloc(len, KM_SLEEP);
845 845 rc = i40e_aq_discover_capabilities(hw, buf, len,
846 846 &needed, i40e_aqc_opc_list_func_capabilities, NULL);
847 847 kmem_free(buf, len);
848 848
849 849 if (hw->aq.asq_last_status == I40E_AQ_RC_ENOMEM &&
850 850 nelems == I40E_HW_CAP_DEFAULT) {
851 851 if (nelems == needed) {
852 852 i40e_error(i40e, "Capability discovery failed "
853 853 "due to byzantine common code");
854 854 return (B_FALSE);
855 855 }
856 856 len = needed;
857 857 continue;
858 858 } else if (rc != I40E_SUCCESS ||
859 859 hw->aq.asq_last_status != I40E_AQ_RC_OK) {
860 860 i40e_error(i40e, "Capability discovery failed: %d", rc);
861 861 return (B_FALSE);
862 862 }
863 863
864 864 break;
865 865 }
866 866
867 867 return (B_TRUE);
868 868 }
869 869
870 870 /*
871 871 * Obtain the switch's capabilities as seen by this PF and keep it around for
872 872 * our later use.
873 873 */
874 874 static boolean_t
875 875 i40e_get_switch_resources(i40e_t *i40e)
876 876 {
877 877 i40e_hw_t *hw = &i40e->i40e_hw_space;
878 878 uint8_t cnt = 2;
879 879 uint8_t act;
880 880 size_t size;
881 881 i40e_switch_rsrc_t *buf;
882 882
883 883 for (;;) {
884 884 enum i40e_status_code ret;
885 885 size = cnt * sizeof (i40e_switch_rsrc_t);
886 886 ASSERT(size > 0);
887 887 if (size > UINT16_MAX)
888 888 return (B_FALSE);
889 889 buf = kmem_alloc(size, KM_SLEEP);
890 890
891 891 ret = i40e_aq_get_switch_resource_alloc(hw, &act, buf,
892 892 cnt, NULL);
893 893 if (ret == I40E_ERR_ADMIN_QUEUE_ERROR &&
894 894 hw->aq.asq_last_status == I40E_AQ_RC_EINVAL) {
895 895 kmem_free(buf, size);
896 896 cnt += I40E_SWITCH_CAP_DEFAULT;
897 897 continue;
898 898 } else if (ret != I40E_SUCCESS) {
899 899 kmem_free(buf, size);
900 900 i40e_error(i40e,
901 901 "failed to retrieve switch statistics: %d", ret);
902 902 return (B_FALSE);
903 903 }
904 904
905 905 break;
906 906 }
907 907
908 908 i40e->i40e_switch_rsrc_alloc = cnt;
909 909 i40e->i40e_switch_rsrc_actual = act;
910 910 i40e->i40e_switch_rsrcs = buf;
911 911
912 912 return (B_TRUE);
913 913 }
914 914
915 915 static void
916 916 i40e_cleanup_resources(i40e_t *i40e)
917 917 {
918 918 if (i40e->i40e_uaddrs != NULL) {
919 919 kmem_free(i40e->i40e_uaddrs, sizeof (i40e_uaddr_t) *
920 920 i40e->i40e_resources.ifr_nmacfilt);
921 921 i40e->i40e_uaddrs = NULL;
922 922 }
923 923
924 924 if (i40e->i40e_maddrs != NULL) {
925 925 kmem_free(i40e->i40e_maddrs, sizeof (i40e_maddr_t) *
926 926 i40e->i40e_resources.ifr_nmcastfilt);
927 927 i40e->i40e_maddrs = NULL;
928 928 }
929 929
930 930 if (i40e->i40e_switch_rsrcs != NULL) {
931 931 size_t sz = sizeof (i40e_switch_rsrc_t) *
932 932 i40e->i40e_switch_rsrc_alloc;
933 933 ASSERT(sz > 0);
934 934 kmem_free(i40e->i40e_switch_rsrcs, sz);
935 935 i40e->i40e_switch_rsrcs = NULL;
936 936 }
937 937
938 938 if (i40e->i40e_device != NULL)
939 939 i40e_device_rele(i40e);
940 940 }
941 941
942 942 static boolean_t
943 943 i40e_get_available_resources(i40e_t *i40e)
944 944 {
945 945 dev_info_t *parent;
946 946 uint16_t bus, device, func;
947 947 uint_t nregs;
948 948 int *regs, i;
949 949 i40e_device_t *idp;
950 950 i40e_hw_t *hw = &i40e->i40e_hw_space;
951 951
952 952 parent = ddi_get_parent(i40e->i40e_dip);
953 953
954 954 if (ddi_prop_lookup_int_array(DDI_DEV_T_ANY, i40e->i40e_dip, 0, "reg",
955 955 ®s, &nregs) != DDI_PROP_SUCCESS) {
956 956 return (B_FALSE);
957 957 }
958 958
959 959 if (nregs < 1) {
960 960 ddi_prop_free(regs);
961 961 return (B_FALSE);
962 962 }
963 963
964 964 bus = PCI_REG_BUS_G(regs[0]);
965 965 device = PCI_REG_DEV_G(regs[0]);
966 966 func = PCI_REG_FUNC_G(regs[0]);
967 967 ddi_prop_free(regs);
968 968
969 969 i40e->i40e_hw_space.bus.func = func;
970 970 i40e->i40e_hw_space.bus.device = device;
971 971
972 972 if (i40e_get_switch_resources(i40e) == B_FALSE) {
973 973 return (B_FALSE);
974 974 }
975 975
976 976 /*
977 977 * To calculate the total amount of a resource we have available, we
978 978 * need to add how many our i40e_t thinks it has guaranteed, if any, and
979 979 * then we need to go through and divide the number of available on the
980 980 * device, which was snapshotted before anyone should have allocated
981 981 * anything, and use that to derive how many are available from the
982 982 * pool. Longer term, we may want to turn this into something that's
983 983 * more of a pool-like resource that everything can share (though that
984 984 * may require some more assistance from MAC).
985 985 *
986 986 * Though for transmit and receive queue pairs, we just have to ask
987 987 * firmware instead.
988 988 */
989 989 idp = i40e_device_find(i40e, parent, bus, device);
990 990 i40e->i40e_device = idp;
991 991 i40e->i40e_resources.ifr_nvsis = 0;
992 992 i40e->i40e_resources.ifr_nvsis_used = 0;
993 993 i40e->i40e_resources.ifr_nmacfilt = 0;
994 994 i40e->i40e_resources.ifr_nmacfilt_used = 0;
995 995 i40e->i40e_resources.ifr_nmcastfilt = 0;
996 996 i40e->i40e_resources.ifr_nmcastfilt_used = 0;
997 997
998 998 for (i = 0; i < i40e->i40e_switch_rsrc_actual; i++) {
999 999 i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i];
1000 1000
1001 1001 switch (srp->resource_type) {
1002 1002 case I40E_AQ_RESOURCE_TYPE_VSI:
1003 1003 i40e->i40e_resources.ifr_nvsis +=
1004 1004 LE_16(srp->guaranteed);
1005 1005 i40e->i40e_resources.ifr_nvsis_used = LE_16(srp->used);
1006 1006 break;
1007 1007 case I40E_AQ_RESOURCE_TYPE_MACADDR:
1008 1008 i40e->i40e_resources.ifr_nmacfilt +=
1009 1009 LE_16(srp->guaranteed);
1010 1010 i40e->i40e_resources.ifr_nmacfilt_used =
1011 1011 LE_16(srp->used);
1012 1012 break;
1013 1013 case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH:
1014 1014 i40e->i40e_resources.ifr_nmcastfilt +=
1015 1015 LE_16(srp->guaranteed);
1016 1016 i40e->i40e_resources.ifr_nmcastfilt_used =
1017 1017 LE_16(srp->used);
1018 1018 break;
1019 1019 default:
1020 1020 break;
1021 1021 }
1022 1022 }
1023 1023
1024 1024 for (i = 0; i < idp->id_rsrcs_act; i++) {
1025 1025 i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i];
1026 1026 switch (srp->resource_type) {
1027 1027 case I40E_AQ_RESOURCE_TYPE_VSI:
1028 1028 i40e->i40e_resources.ifr_nvsis +=
1029 1029 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1030 1030 break;
1031 1031 case I40E_AQ_RESOURCE_TYPE_MACADDR:
1032 1032 i40e->i40e_resources.ifr_nmacfilt +=
1033 1033 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1034 1034 break;
1035 1035 case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH:
1036 1036 i40e->i40e_resources.ifr_nmcastfilt +=
1037 1037 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1038 1038 default:
1039 1039 break;
1040 1040 }
1041 1041 }
1042 1042
1043 1043 i40e->i40e_resources.ifr_nrx_queue = hw->func_caps.num_rx_qp;
1044 1044 i40e->i40e_resources.ifr_ntx_queue = hw->func_caps.num_tx_qp;
1045 1045
1046 1046 i40e->i40e_uaddrs = kmem_zalloc(sizeof (i40e_uaddr_t) *
1047 1047 i40e->i40e_resources.ifr_nmacfilt, KM_SLEEP);
1048 1048 i40e->i40e_maddrs = kmem_zalloc(sizeof (i40e_maddr_t) *
1049 1049 i40e->i40e_resources.ifr_nmcastfilt, KM_SLEEP);
1050 1050
1051 1051 /*
1052 1052 * Initialize these as multicast addresses to indicate it's invalid for
1053 1053 * sanity purposes. Think of it like 0xdeadbeef.
1054 1054 */
1055 1055 for (i = 0; i < i40e->i40e_resources.ifr_nmacfilt; i++)
1056 1056 i40e->i40e_uaddrs[i].iua_mac[0] = 0x01;
1057 1057
1058 1058 return (B_TRUE);
1059 1059 }
1060 1060
1061 1061 static boolean_t
1062 1062 i40e_enable_interrupts(i40e_t *i40e)
1063 1063 {
1064 1064 int i, rc;
1065 1065
1066 1066 if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) {
1067 1067 rc = ddi_intr_block_enable(i40e->i40e_intr_handles,
1068 1068 i40e->i40e_intr_count);
1069 1069 if (rc != DDI_SUCCESS) {
1070 1070 i40e_error(i40e, "Interrupt block-enable failed: %d",
1071 1071 rc);
1072 1072 return (B_FALSE);
1073 1073 }
1074 1074 } else {
1075 1075 for (i = 0; i < i40e->i40e_intr_count; i++) {
1076 1076 rc = ddi_intr_enable(i40e->i40e_intr_handles[i]);
1077 1077 if (rc != DDI_SUCCESS) {
1078 1078 i40e_error(i40e,
1079 1079 "Failed to enable interrupt %d: %d", i, rc);
1080 1080 while (--i >= 0) {
1081 1081 (void) ddi_intr_disable(
1082 1082 i40e->i40e_intr_handles[i]);
1083 1083 }
1084 1084 return (B_FALSE);
1085 1085 }
1086 1086 }
1087 1087 }
1088 1088
1089 1089 return (B_TRUE);
1090 1090 }
1091 1091
1092 1092 static boolean_t
1093 1093 i40e_disable_interrupts(i40e_t *i40e)
1094 1094 {
1095 1095 int i, rc;
1096 1096
1097 1097 if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) {
1098 1098 rc = ddi_intr_block_disable(i40e->i40e_intr_handles,
1099 1099 i40e->i40e_intr_count);
1100 1100 if (rc != DDI_SUCCESS) {
1101 1101 i40e_error(i40e,
1102 1102 "Interrupt block-disabled failed: %d", rc);
1103 1103 return (B_FALSE);
1104 1104 }
1105 1105 } else {
1106 1106 for (i = 0; i < i40e->i40e_intr_count; i++) {
1107 1107 rc = ddi_intr_disable(i40e->i40e_intr_handles[i]);
1108 1108 if (rc != DDI_SUCCESS) {
1109 1109 i40e_error(i40e,
1110 1110 "Failed to disable interrupt %d: %d",
1111 1111 i, rc);
1112 1112 return (B_FALSE);
1113 1113 }
1114 1114 }
1115 1115 }
1116 1116
1117 1117 return (B_TRUE);
1118 1118 }
1119 1119
1120 1120 /*
1121 1121 * Free receive & transmit rings.
1122 1122 */
1123 1123 static void
1124 1124 i40e_free_trqpairs(i40e_t *i40e)
1125 1125 {
1126 1126 i40e_trqpair_t *itrq;
1127 1127
1128 1128 if (i40e->i40e_rx_groups != NULL) {
1129 1129 kmem_free(i40e->i40e_rx_groups,
1130 1130 sizeof (i40e_rx_group_t) * i40e->i40e_num_rx_groups);
1131 1131 i40e->i40e_rx_groups = NULL;
1132 1132 }
1133 1133
1134 1134 if (i40e->i40e_trqpairs != NULL) {
1135 1135 for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1136 1136 itrq = &i40e->i40e_trqpairs[i];
1137 1137 mutex_destroy(&itrq->itrq_rx_lock);
1138 1138 mutex_destroy(&itrq->itrq_tx_lock);
1139 1139 mutex_destroy(&itrq->itrq_tcb_lock);
1140 1140
1141 1141 /*
1142 1142 * Should have already been cleaned up by start/stop,
1143 1143 * etc.
1144 1144 */
1145 1145 ASSERT(itrq->itrq_txkstat == NULL);
1146 1146 ASSERT(itrq->itrq_rxkstat == NULL);
1147 1147 }
1148 1148
1149 1149 kmem_free(i40e->i40e_trqpairs,
1150 1150 sizeof (i40e_trqpair_t) * i40e->i40e_num_trqpairs);
1151 1151 i40e->i40e_trqpairs = NULL;
1152 1152 }
1153 1153
1154 1154 cv_destroy(&i40e->i40e_rx_pending_cv);
1155 1155 mutex_destroy(&i40e->i40e_rx_pending_lock);
1156 1156 mutex_destroy(&i40e->i40e_general_lock);
1157 1157 }
1158 1158
1159 1159 /*
1160 1160 * Allocate transmit and receive rings, as well as other data structures that we
1161 1161 * need.
1162 1162 */
1163 1163 static boolean_t
1164 1164 i40e_alloc_trqpairs(i40e_t *i40e)
1165 1165 {
1166 1166 void *mutexpri = DDI_INTR_PRI(i40e->i40e_intr_pri);
1167 1167
1168 1168 /*
1169 1169 * Now that we have the priority for the interrupts, initialize
1170 1170 * all relevant locks.
1171 1171 */
1172 1172 mutex_init(&i40e->i40e_general_lock, NULL, MUTEX_DRIVER, mutexpri);
1173 1173 mutex_init(&i40e->i40e_rx_pending_lock, NULL, MUTEX_DRIVER, mutexpri);
1174 1174 cv_init(&i40e->i40e_rx_pending_cv, NULL, CV_DRIVER, NULL);
1175 1175
1176 1176 i40e->i40e_trqpairs = kmem_zalloc(sizeof (i40e_trqpair_t) *
1177 1177 i40e->i40e_num_trqpairs, KM_SLEEP);
1178 1178 for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1179 1179 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
1180 1180
1181 1181 itrq->itrq_i40e = i40e;
1182 1182 mutex_init(&itrq->itrq_rx_lock, NULL, MUTEX_DRIVER, mutexpri);
1183 1183 mutex_init(&itrq->itrq_tx_lock, NULL, MUTEX_DRIVER, mutexpri);
1184 1184 mutex_init(&itrq->itrq_tcb_lock, NULL, MUTEX_DRIVER, mutexpri);
1185 1185 itrq->itrq_index = i;
1186 1186 }
1187 1187
1188 1188 i40e->i40e_rx_groups = kmem_zalloc(sizeof (i40e_rx_group_t) *
1189 1189 i40e->i40e_num_rx_groups, KM_SLEEP);
1190 1190
1191 1191 for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) {
1192 1192 i40e_rx_group_t *rxg = &i40e->i40e_rx_groups[i];
1193 1193
1194 1194 rxg->irg_index = i;
1195 1195 rxg->irg_i40e = i40e;
1196 1196 }
1197 1197
1198 1198 return (B_TRUE);
1199 1199 }
1200 1200
1201 1201
1202 1202
1203 1203 /*
1204 1204 * Unless a .conf file already overrode i40e_t structure values, they will
1205 1205 * be 0, and need to be set in conjunction with the now-available HW report.
1206 1206 */
1207 1207 /* ARGSUSED */
1208 1208 static void
1209 1209 i40e_hw_to_instance(i40e_t *i40e, i40e_hw_t *hw)
1210 1210 {
1211 1211 if (i40e->i40e_num_trqpairs_per_vsi == 0) {
1212 1212 if (i40e_is_x722(i40e)) {
1213 1213 i40e->i40e_num_trqpairs_per_vsi =
1214 1214 I40E_722_MAX_TC_QUEUES;
1215 1215 } else {
1216 1216 i40e->i40e_num_trqpairs_per_vsi =
1217 1217 I40E_710_MAX_TC_QUEUES;
1218 1218 }
1219 1219 }
1220 1220
1221 1221 if (i40e->i40e_num_rx_groups == 0) {
1222 1222 i40e->i40e_num_rx_groups = I40E_GROUP_MAX;
1223 1223 }
1224 1224 }
1225 1225
1226 1226 /*
1227 1227 * Free any resources required by, or setup by, the Intel common code.
1228 1228 */
1229 1229 static void
1230 1230 i40e_common_code_fini(i40e_t *i40e)
1231 1231 {
1232 1232 i40e_hw_t *hw = &i40e->i40e_hw_space;
1233 1233 int rc;
1234 1234
1235 1235 rc = i40e_shutdown_lan_hmc(hw);
1236 1236 if (rc != I40E_SUCCESS)
1237 1237 i40e_error(i40e, "failed to shutdown LAN hmc: %d", rc);
1238 1238
1239 1239 rc = i40e_shutdown_adminq(hw);
1240 1240 if (rc != I40E_SUCCESS)
1241 1241 i40e_error(i40e, "failed to shutdown admin queue: %d", rc);
1242 1242 }
1243 1243
1244 1244 /*
1245 1245 * Initialize and call Intel common-code routines, includes some setup
1246 1246 * the common code expects from the driver. Also prints on failure, so
1247 1247 * the caller doesn't have to.
1248 1248 */
1249 1249 static boolean_t
1250 1250 i40e_common_code_init(i40e_t *i40e, i40e_hw_t *hw)
1251 1251 {
1252 1252 int rc;
1253 1253
1254 1254 i40e_clear_hw(hw);
1255 1255 rc = i40e_pf_reset(hw);
1256 1256 if (rc != 0) {
1257 1257 i40e_error(i40e, "failed to reset hardware: %d", rc);
1258 1258 i40e_fm_ereport(i40e, DDI_FM_DEVICE_NO_RESPONSE);
1259 1259 return (B_FALSE);
1260 1260 }
1261 1261
1262 1262 rc = i40e_init_shared_code(hw);
1263 1263 if (rc != 0) {
1264 1264 i40e_error(i40e, "failed to initialize i40e core: %d", rc);
1265 1265 return (B_FALSE);
1266 1266 }
1267 1267
1268 1268 hw->aq.num_arq_entries = I40E_DEF_ADMINQ_SIZE;
1269 1269 hw->aq.num_asq_entries = I40E_DEF_ADMINQ_SIZE;
1270 1270 hw->aq.arq_buf_size = I40E_ADMINQ_BUFSZ;
1271 1271 hw->aq.asq_buf_size = I40E_ADMINQ_BUFSZ;
1272 1272
1273 1273 rc = i40e_init_adminq(hw);
1274 1274 if (rc != 0) {
1275 1275 i40e_error(i40e, "failed to initialize firmware admin queue: "
1276 1276 "%d, potential firmware version mismatch", rc);
1277 1277 i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE);
1278 1278 return (B_FALSE);
1279 1279 }
1280 1280
1281 1281 if (hw->aq.api_maj_ver == I40E_FW_API_VERSION_MAJOR &&
1282 1282 hw->aq.api_min_ver > I40E_FW_API_VERSION_MINOR) {
1283 1283 i40e_log(i40e, "The driver for the device detected a newer "
1284 1284 "version of the NVM image (%d.%d) than expected (%d.%d).\n"
1285 1285 "Please install the most recent version of the network "
1286 1286 "driver.\n", hw->aq.api_maj_ver, hw->aq.api_min_ver,
1287 1287 I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR);
1288 1288 } else if (hw->aq.api_maj_ver < I40E_FW_API_VERSION_MAJOR ||
1289 1289 hw->aq.api_min_ver < (I40E_FW_API_VERSION_MINOR - 1)) {
1290 1290 i40e_log(i40e, "The driver for the device detected an older"
1291 1291 " version of the NVM image (%d.%d) than expected (%d.%d)."
1292 1292 "\nPlease update the NVM image.\n",
1293 1293 hw->aq.api_maj_ver, hw->aq.api_min_ver,
1294 1294 I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR - 1);
1295 1295 }
1296 1296
1297 1297 i40e_clear_pxe_mode(hw);
1298 1298
1299 1299 /*
1300 1300 * We need to call this so that the common code can discover
1301 1301 * capabilities of the hardware, which it uses throughout the rest.
1302 1302 */
1303 1303 if (!i40e_get_hw_capabilities(i40e, hw)) {
1304 1304 i40e_error(i40e, "failed to obtain hardware capabilities");
1305 1305 return (B_FALSE);
1306 1306 }
1307 1307
1308 1308 if (i40e_get_available_resources(i40e) == B_FALSE) {
1309 1309 i40e_error(i40e, "failed to obtain hardware resources");
1310 1310 return (B_FALSE);
1311 1311 }
1312 1312
1313 1313 i40e_hw_to_instance(i40e, hw);
1314 1314
1315 1315 rc = i40e_init_lan_hmc(hw, hw->func_caps.num_tx_qp,
1316 1316 hw->func_caps.num_rx_qp, 0, 0);
1317 1317 if (rc != 0) {
1318 1318 i40e_error(i40e, "failed to initialize hardware memory cache: "
1319 1319 "%d", rc);
1320 1320 return (B_FALSE);
1321 1321 }
1322 1322
1323 1323 rc = i40e_configure_lan_hmc(hw, I40E_HMC_MODEL_DIRECT_ONLY);
1324 1324 if (rc != 0) {
1325 1325 i40e_error(i40e, "failed to configure hardware memory cache: "
1326 1326 "%d", rc);
1327 1327 return (B_FALSE);
1328 1328 }
1329 1329
1330 1330 (void) i40e_aq_stop_lldp(hw, TRUE, NULL);
1331 1331
1332 1332 rc = i40e_get_mac_addr(hw, hw->mac.addr);
1333 1333 if (rc != I40E_SUCCESS) {
1334 1334 i40e_error(i40e, "failed to retrieve hardware mac address: %d",
1335 1335 rc);
1336 1336 return (B_FALSE);
1337 1337 }
1338 1338
1339 1339 rc = i40e_validate_mac_addr(hw->mac.addr);
1340 1340 if (rc != 0) {
1341 1341 i40e_error(i40e, "failed to validate internal mac address: "
1342 1342 "%d", rc);
1343 1343 return (B_FALSE);
1344 1344 }
1345 1345 bcopy(hw->mac.addr, hw->mac.perm_addr, ETHERADDRL);
1346 1346 if ((rc = i40e_get_port_mac_addr(hw, hw->mac.port_addr)) !=
1347 1347 I40E_SUCCESS) {
1348 1348 i40e_error(i40e, "failed to retrieve port mac address: %d",
1349 1349 rc);
1350 1350 return (B_FALSE);
1351 1351 }
1352 1352
1353 1353 /*
1354 1354 * We need to obtain the Default Virtual Station SEID (VSI)
1355 1355 * before we can perform other operations on the device.
1356 1356 */
1357 1357 if (!i40e_set_def_vsi_seid(i40e)) {
1358 1358 i40e_error(i40e, "failed to obtain Default VSI SEID");
1359 1359 return (B_FALSE);
1360 1360 }
1361 1361
1362 1362 return (B_TRUE);
1363 1363 }
1364 1364
1365 1365 static void
1366 1366 i40e_unconfigure(dev_info_t *devinfo, i40e_t *i40e)
1367 1367 {
1368 1368 int rc;
1369 1369
1370 1370 if (i40e->i40e_attach_progress & I40E_ATTACH_ENABLE_INTR)
1371 1371 (void) i40e_disable_interrupts(i40e);
1372 1372
1373 1373 if ((i40e->i40e_attach_progress & I40E_ATTACH_LINK_TIMER) &&
1374 1374 i40e->i40e_periodic_id != 0) {
1375 1375 ddi_periodic_delete(i40e->i40e_periodic_id);
1376 1376 i40e->i40e_periodic_id = 0;
1377 1377 }
1378 1378
1379 1379 if (i40e->i40e_attach_progress & I40E_ATTACH_MAC) {
1380 1380 rc = mac_unregister(i40e->i40e_mac_hdl);
1381 1381 if (rc != 0) {
1382 1382 i40e_error(i40e, "failed to unregister from mac: %d",
1383 1383 rc);
1384 1384 }
1385 1385 }
1386 1386
1387 1387 if (i40e->i40e_attach_progress & I40E_ATTACH_STATS) {
1388 1388 i40e_stats_fini(i40e);
1389 1389 }
1390 1390
1391 1391 if (i40e->i40e_attach_progress & I40E_ATTACH_ADD_INTR)
1392 1392 i40e_rem_intr_handlers(i40e);
1393 1393
1394 1394 if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_RINGSLOCKS)
1395 1395 i40e_free_trqpairs(i40e);
1396 1396
1397 1397 if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_INTR)
1398 1398 i40e_rem_intrs(i40e);
1399 1399
1400 1400 if (i40e->i40e_attach_progress & I40E_ATTACH_COMMON_CODE)
1401 1401 i40e_common_code_fini(i40e);
1402 1402
1403 1403 i40e_cleanup_resources(i40e);
1404 1404
1405 1405 if (i40e->i40e_attach_progress & I40E_ATTACH_PROPS)
1406 1406 (void) ddi_prop_remove_all(devinfo);
1407 1407
1408 1408 if (i40e->i40e_attach_progress & I40E_ATTACH_REGS_MAP &&
1409 1409 i40e->i40e_osdep_space.ios_reg_handle != NULL) {
1410 1410 ddi_regs_map_free(&i40e->i40e_osdep_space.ios_reg_handle);
1411 1411 i40e->i40e_osdep_space.ios_reg_handle = NULL;
1412 1412 }
1413 1413
1414 1414 if ((i40e->i40e_attach_progress & I40E_ATTACH_PCI_CONFIG) &&
1415 1415 i40e->i40e_osdep_space.ios_cfg_handle != NULL) {
1416 1416 pci_config_teardown(&i40e->i40e_osdep_space.ios_cfg_handle);
1417 1417 i40e->i40e_osdep_space.ios_cfg_handle = NULL;
1418 1418 }
1419 1419
1420 1420 if (i40e->i40e_attach_progress & I40E_ATTACH_FM_INIT)
1421 1421 i40e_fm_fini(i40e);
1422 1422
1423 1423 if (i40e->i40e_attach_progress & I40E_ATTACH_UFM_INIT)
1424 1424 ddi_ufm_fini(i40e->i40e_ufmh);
1425 1425
1426 1426 kmem_free(i40e->i40e_aqbuf, I40E_ADMINQ_BUFSZ);
1427 1427 kmem_free(i40e, sizeof (i40e_t));
1428 1428
1429 1429 ddi_set_driver_private(devinfo, NULL);
1430 1430 }
1431 1431
1432 1432 static boolean_t
1433 1433 i40e_final_init(i40e_t *i40e)
1434 1434 {
1435 1435 i40e_hw_t *hw = &i40e->i40e_hw_space;
1436 1436 struct i40e_osdep *osdep = OS_DEP(hw);
1437 1437 uint8_t pbanum[I40E_PBANUM_STRLEN];
1438 1438 enum i40e_status_code irc;
1439 1439 char buf[I40E_DDI_PROP_LEN];
1440 1440
1441 1441 pbanum[0] = '\0';
1442 1442 irc = i40e_read_pba_string(hw, pbanum, sizeof (pbanum));
1443 1443 if (irc != I40E_SUCCESS) {
1444 1444 i40e_log(i40e, "failed to read PBA string: %d", irc);
1445 1445 } else {
1446 1446 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1447 1447 "printed-board-assembly", (char *)pbanum);
1448 1448 }
1449 1449
1450 1450 #ifdef DEBUG
1451 1451 ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.fw_maj_ver,
1452 1452 hw->aq.fw_min_ver) < sizeof (buf));
1453 1453 ASSERT(snprintf(NULL, 0, "%x", hw->aq.fw_build) < sizeof (buf));
1454 1454 ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.api_maj_ver,
1455 1455 hw->aq.api_min_ver) < sizeof (buf));
1456 1456 #endif
1457 1457
1458 1458 (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.fw_maj_ver,
1459 1459 hw->aq.fw_min_ver);
1460 1460 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1461 1461 "firmware-version", buf);
1462 1462 (void) snprintf(buf, sizeof (buf), "%x", hw->aq.fw_build);
1463 1463 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1464 1464 "firmware-build", buf);
1465 1465 (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.api_maj_ver,
1466 1466 hw->aq.api_min_ver);
1467 1467 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1468 1468 "api-version", buf);
1469 1469
1470 1470 if (!i40e_set_hw_bus_info(hw))
1471 1471 return (B_FALSE);
1472 1472
1473 1473 if (i40e_check_acc_handle(osdep->ios_reg_handle) != DDI_FM_OK) {
1474 1474 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
1475 1475 return (B_FALSE);
1476 1476 }
1477 1477
1478 1478 return (B_TRUE);
1479 1479 }
1480 1480
1481 1481 static void
1482 1482 i40e_identify_hardware(i40e_t *i40e)
1483 1483 {
1484 1484 i40e_hw_t *hw = &i40e->i40e_hw_space;
1485 1485 struct i40e_osdep *osdep = &i40e->i40e_osdep_space;
1486 1486
1487 1487 hw->vendor_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_VENID);
1488 1488 hw->device_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_DEVID);
1489 1489 hw->revision_id = pci_config_get8(osdep->ios_cfg_handle,
1490 1490 PCI_CONF_REVID);
1491 1491 hw->subsystem_device_id =
1492 1492 pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBSYSID);
1493 1493 hw->subsystem_vendor_id =
1494 1494 pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBVENID);
1495 1495
1496 1496 /*
1497 1497 * Note that we set the hardware's bus information later on, in
1498 1498 * i40e_get_available_resources(). The common code doesn't seem to
1499 1499 * require that it be set in any ways, it seems to be mostly for
1500 1500 * book-keeping.
1501 1501 */
1502 1502 }
1503 1503
1504 1504 static boolean_t
1505 1505 i40e_regs_map(i40e_t *i40e)
1506 1506 {
1507 1507 dev_info_t *devinfo = i40e->i40e_dip;
1508 1508 i40e_hw_t *hw = &i40e->i40e_hw_space;
1509 1509 struct i40e_osdep *osdep = &i40e->i40e_osdep_space;
1510 1510 off_t memsize;
1511 1511 int ret;
1512 1512
1513 1513 if (ddi_dev_regsize(devinfo, I40E_ADAPTER_REGSET, &memsize) !=
1514 1514 DDI_SUCCESS) {
1515 1515 i40e_error(i40e, "Used invalid register set to map PCIe regs");
1516 1516 return (B_FALSE);
1517 1517 }
1518 1518
1519 1519 if ((ret = ddi_regs_map_setup(devinfo, I40E_ADAPTER_REGSET,
1520 1520 (caddr_t *)&hw->hw_addr, 0, memsize, &i40e_regs_acc_attr,
1521 1521 &osdep->ios_reg_handle)) != DDI_SUCCESS) {
1522 1522 i40e_error(i40e, "failed to map device registers: %d", ret);
1523 1523 return (B_FALSE);
1524 1524 }
1525 1525
1526 1526 osdep->ios_reg_size = memsize;
1527 1527 return (B_TRUE);
1528 1528 }
1529 1529
1530 1530 /*
1531 1531 * Update parameters required when a new MTU has been configured. Calculate the
1532 1532 * maximum frame size, as well as, size our DMA buffers which we size in
1533 1533 * increments of 1K.
1534 1534 */
1535 1535 void
1536 1536 i40e_update_mtu(i40e_t *i40e)
1537 1537 {
1538 1538 uint32_t rx, tx;
1539 1539
1540 1540 i40e->i40e_frame_max = i40e->i40e_sdu +
1541 1541 sizeof (struct ether_vlan_header) + ETHERFCSL;
1542 1542
1543 1543 rx = i40e->i40e_frame_max + I40E_BUF_IPHDR_ALIGNMENT;
1544 1544 i40e->i40e_rx_buf_size = ((rx >> 10) +
1545 1545 ((rx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10;
1546 1546
1547 1547 tx = i40e->i40e_frame_max;
1548 1548 i40e->i40e_tx_buf_size = ((tx >> 10) +
1549 1549 ((tx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10;
1550 1550 }
1551 1551
1552 1552 static int
1553 1553 i40e_get_prop(i40e_t *i40e, char *prop, int min, int max, int def)
1554 1554 {
1555 1555 int val;
1556 1556
1557 1557 val = ddi_prop_get_int(DDI_DEV_T_ANY, i40e->i40e_dip, DDI_PROP_DONTPASS,
1558 1558 prop, def);
1559 1559 if (val > max)
1560 1560 val = max;
1561 1561 if (val < min)
1562 1562 val = min;
1563 1563 return (val);
1564 1564 }
1565 1565
1566 1566 static void
1567 1567 i40e_init_properties(i40e_t *i40e)
1568 1568 {
1569 1569 i40e->i40e_sdu = i40e_get_prop(i40e, "default_mtu",
1570 1570 I40E_MIN_MTU, I40E_MAX_MTU, I40E_DEF_MTU);
1571 1571
1572 1572 i40e->i40e_intr_force = i40e_get_prop(i40e, "intr_force",
1573 1573 I40E_INTR_NONE, I40E_INTR_LEGACY, I40E_INTR_NONE);
1574 1574
1575 1575 i40e->i40e_mr_enable = i40e_get_prop(i40e, "mr_enable",
1576 1576 B_FALSE, B_TRUE, B_TRUE);
1577 1577
1578 1578 i40e->i40e_tx_ring_size = i40e_get_prop(i40e, "tx_ring_size",
1579 1579 I40E_MIN_TX_RING_SIZE, I40E_MAX_TX_RING_SIZE,
1580 1580 I40E_DEF_TX_RING_SIZE);
1581 1581 if ((i40e->i40e_tx_ring_size % I40E_DESC_ALIGN) != 0) {
1582 1582 i40e->i40e_tx_ring_size = P2ROUNDUP(i40e->i40e_tx_ring_size,
1583 1583 I40E_DESC_ALIGN);
1584 1584 }
1585 1585
1586 1586 i40e->i40e_tx_block_thresh = i40e_get_prop(i40e, "tx_resched_threshold",
1587 1587 I40E_MIN_TX_BLOCK_THRESH,
1588 1588 i40e->i40e_tx_ring_size - I40E_TX_MAX_COOKIE,
1589 1589 I40E_DEF_TX_BLOCK_THRESH);
1590 1590
1591 1591 i40e->i40e_rx_ring_size = i40e_get_prop(i40e, "rx_ring_size",
1592 1592 I40E_MIN_RX_RING_SIZE, I40E_MAX_RX_RING_SIZE,
1593 1593 I40E_DEF_RX_RING_SIZE);
1594 1594 if ((i40e->i40e_rx_ring_size % I40E_DESC_ALIGN) != 0) {
1595 1595 i40e->i40e_rx_ring_size = P2ROUNDUP(i40e->i40e_rx_ring_size,
1596 1596 I40E_DESC_ALIGN);
1597 1597 }
1598 1598
1599 1599 i40e->i40e_rx_limit_per_intr = i40e_get_prop(i40e, "rx_limit_per_intr",
1600 1600 I40E_MIN_RX_LIMIT_PER_INTR, I40E_MAX_RX_LIMIT_PER_INTR,
1601 1601 I40E_DEF_RX_LIMIT_PER_INTR);
1602 1602
1603 1603 i40e->i40e_tx_hcksum_enable = i40e_get_prop(i40e, "tx_hcksum_enable",
1604 1604 B_FALSE, B_TRUE, B_TRUE);
1605 1605
1606 1606 i40e->i40e_tx_lso_enable = i40e_get_prop(i40e, "tx_lso_enable",
1607 1607 B_FALSE, B_TRUE, B_TRUE);
1608 1608
1609 1609 i40e->i40e_rx_hcksum_enable = i40e_get_prop(i40e, "rx_hcksum_enable",
1610 1610 B_FALSE, B_TRUE, B_TRUE);
1611 1611
1612 1612 i40e->i40e_rx_dma_min = i40e_get_prop(i40e, "rx_dma_threshold",
1613 1613 I40E_MIN_RX_DMA_THRESH, I40E_MAX_RX_DMA_THRESH,
1614 1614 I40E_DEF_RX_DMA_THRESH);
1615 1615
1616 1616 i40e->i40e_tx_dma_min = i40e_get_prop(i40e, "tx_dma_threshold",
1617 1617 I40E_MIN_TX_DMA_THRESH, I40E_MAX_TX_DMA_THRESH,
1618 1618 I40E_DEF_TX_DMA_THRESH);
1619 1619
1620 1620 i40e->i40e_tx_itr = i40e_get_prop(i40e, "tx_intr_throttle",
1621 1621 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_TX_ITR);
1622 1622
1623 1623 i40e->i40e_rx_itr = i40e_get_prop(i40e, "rx_intr_throttle",
1624 1624 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_RX_ITR);
1625 1625
1626 1626 i40e->i40e_other_itr = i40e_get_prop(i40e, "other_intr_throttle",
1627 1627 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_OTHER_ITR);
1628 1628
1629 1629 if (!i40e->i40e_mr_enable) {
1630 1630 i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX;
1631 1631 i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX;
1632 1632 }
1633 1633
1634 1634 i40e_update_mtu(i40e);
1635 1635 }
1636 1636
1637 1637 /*
1638 1638 * There are a few constraints on interrupts that we're currently imposing, some
1639 1639 * of which are restrictions from hardware. For a fuller treatment, see
1640 1640 * i40e_intr.c.
1641 1641 *
1642 1642 * Currently, to use MSI-X we require two interrupts be available though in
1643 1643 * theory we should participate in IRM and happily use more interrupts.
1644 1644 *
1645 1645 * Hardware only supports a single MSI being programmed and therefore if we
1646 1646 * don't have MSI-X interrupts available at this time, then we ratchet down the
1647 1647 * number of rings and groups available. Obviously, we only bother with a single
1648 1648 * fixed interrupt.
1649 1649 */
1650 1650 static boolean_t
1651 1651 i40e_alloc_intr_handles(i40e_t *i40e, dev_info_t *devinfo, int intr_type)
1652 1652 {
1653 1653 i40e_hw_t *hw = &i40e->i40e_hw_space;
1654 1654 ddi_acc_handle_t rh = i40e->i40e_osdep_space.ios_reg_handle;
1655 1655 int request, count, actual, rc, min;
1656 1656 uint32_t reg;
1657 1657
1658 1658 switch (intr_type) {
1659 1659 case DDI_INTR_TYPE_FIXED:
1660 1660 case DDI_INTR_TYPE_MSI:
1661 1661 request = 1;
1662 1662 min = 1;
1663 1663 break;
1664 1664 case DDI_INTR_TYPE_MSIX:
1665 1665 min = 2;
1666 1666 if (!i40e->i40e_mr_enable) {
1667 1667 request = 2;
1668 1668 break;
1669 1669 }
1670 1670 reg = I40E_READ_REG(hw, I40E_GLPCI_CNF2);
1671 1671 /*
1672 1672 * Should this read fail, we will drop back to using
1673 1673 * MSI or fixed interrupts.
1674 1674 */
1675 1675 if (i40e_check_acc_handle(rh) != DDI_FM_OK) {
1676 1676 ddi_fm_service_impact(i40e->i40e_dip,
|
↓ open down ↓ |
1676 lines elided |
↑ open up ↑ |
1677 1677 DDI_SERVICE_DEGRADED);
1678 1678 return (B_FALSE);
1679 1679 }
1680 1680 request = (reg & I40E_GLPCI_CNF2_MSI_X_PF_N_MASK) >>
1681 1681 I40E_GLPCI_CNF2_MSI_X_PF_N_SHIFT;
1682 1682 request++; /* the register value is n - 1 */
1683 1683 break;
1684 1684 default:
1685 1685 panic("bad interrupt type passed to i40e_alloc_intr_handles: "
1686 1686 "%d", intr_type);
1687 - return (B_FALSE);
1688 1687 }
1689 1688
1690 1689 rc = ddi_intr_get_nintrs(devinfo, intr_type, &count);
1691 1690 if (rc != DDI_SUCCESS || count < min) {
1692 1691 i40e_log(i40e, "Get interrupt number failed, "
1693 1692 "returned %d, count %d", rc, count);
1694 1693 return (B_FALSE);
1695 1694 }
1696 1695
1697 1696 rc = ddi_intr_get_navail(devinfo, intr_type, &count);
1698 1697 if (rc != DDI_SUCCESS || count < min) {
1699 1698 i40e_log(i40e, "Get AVAILABLE interrupt number failed, "
1700 1699 "returned %d, count %d", rc, count);
1701 1700 return (B_FALSE);
1702 1701 }
1703 1702
1704 1703 actual = 0;
1705 1704 i40e->i40e_intr_count = 0;
1706 1705 i40e->i40e_intr_count_max = 0;
1707 1706 i40e->i40e_intr_count_min = 0;
1708 1707
1709 1708 i40e->i40e_intr_size = request * sizeof (ddi_intr_handle_t);
1710 1709 ASSERT(i40e->i40e_intr_size != 0);
1711 1710 i40e->i40e_intr_handles = kmem_alloc(i40e->i40e_intr_size, KM_SLEEP);
1712 1711
1713 1712 rc = ddi_intr_alloc(devinfo, i40e->i40e_intr_handles, intr_type, 0,
1714 1713 min(request, count), &actual, DDI_INTR_ALLOC_NORMAL);
1715 1714 if (rc != DDI_SUCCESS) {
1716 1715 i40e_log(i40e, "Interrupt allocation failed with %d.", rc);
1717 1716 goto alloc_handle_fail;
1718 1717 }
1719 1718
1720 1719 i40e->i40e_intr_count = actual;
1721 1720 i40e->i40e_intr_count_max = request;
1722 1721 i40e->i40e_intr_count_min = min;
1723 1722
1724 1723 if (actual < min) {
1725 1724 i40e_log(i40e, "actual (%d) is less than minimum (%d).",
1726 1725 actual, min);
1727 1726 goto alloc_handle_fail;
1728 1727 }
1729 1728
1730 1729 /*
1731 1730 * Record the priority and capabilities for our first vector. Once
1732 1731 * we have it, that's our priority until detach time. Even if we
1733 1732 * eventually participate in IRM, our priority shouldn't change.
1734 1733 */
1735 1734 rc = ddi_intr_get_pri(i40e->i40e_intr_handles[0], &i40e->i40e_intr_pri);
1736 1735 if (rc != DDI_SUCCESS) {
1737 1736 i40e_log(i40e,
1738 1737 "Getting interrupt priority failed with %d.", rc);
1739 1738 goto alloc_handle_fail;
1740 1739 }
1741 1740
1742 1741 rc = ddi_intr_get_cap(i40e->i40e_intr_handles[0], &i40e->i40e_intr_cap);
1743 1742 if (rc != DDI_SUCCESS) {
1744 1743 i40e_log(i40e,
1745 1744 "Getting interrupt capabilities failed with %d.", rc);
1746 1745 goto alloc_handle_fail;
1747 1746 }
1748 1747
1749 1748 i40e->i40e_intr_type = intr_type;
1750 1749 return (B_TRUE);
1751 1750
1752 1751 alloc_handle_fail:
1753 1752
1754 1753 i40e_rem_intrs(i40e);
1755 1754 return (B_FALSE);
1756 1755 }
1757 1756
1758 1757 static boolean_t
1759 1758 i40e_alloc_intrs(i40e_t *i40e, dev_info_t *devinfo)
1760 1759 {
1761 1760 int intr_types, rc;
1762 1761 uint_t max_trqpairs;
1763 1762
1764 1763 if (i40e_is_x722(i40e)) {
1765 1764 max_trqpairs = I40E_722_MAX_TC_QUEUES;
1766 1765 } else {
1767 1766 max_trqpairs = I40E_710_MAX_TC_QUEUES;
1768 1767 }
1769 1768
1770 1769 rc = ddi_intr_get_supported_types(devinfo, &intr_types);
1771 1770 if (rc != DDI_SUCCESS) {
1772 1771 i40e_error(i40e, "failed to get supported interrupt types: %d",
1773 1772 rc);
1774 1773 return (B_FALSE);
1775 1774 }
1776 1775
1777 1776 i40e->i40e_intr_type = 0;
1778 1777 i40e->i40e_num_rx_groups = I40E_GROUP_MAX;
1779 1778
1780 1779 /*
1781 1780 * We need to determine the number of queue pairs per traffic
1782 1781 * class. We only have one traffic class (TC0), so we'll base
1783 1782 * this off the number of interrupts provided. Furthermore,
1784 1783 * since we only use one traffic class, the number of queues
1785 1784 * per traffic class and per VSI are the same.
1786 1785 */
1787 1786 if ((intr_types & DDI_INTR_TYPE_MSIX) &&
1788 1787 (i40e->i40e_intr_force <= I40E_INTR_MSIX) &&
1789 1788 (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSIX))) {
1790 1789 uint32_t n;
1791 1790
1792 1791 /*
1793 1792 * While we want the number of queue pairs to match
1794 1793 * the number of interrupts, we must keep stay in
1795 1794 * bounds of the maximum number of queues per traffic
1796 1795 * class. We subtract one from i40e_intr_count to
1797 1796 * account for interrupt zero; which is currently
1798 1797 * restricted to admin queue commands and other
1799 1798 * interrupt causes.
1800 1799 */
1801 1800 n = MIN(i40e->i40e_intr_count - 1, max_trqpairs);
1802 1801 ASSERT3U(n, >, 0);
1803 1802
1804 1803 /*
1805 1804 * Round up to the nearest power of two to ensure that
1806 1805 * the QBASE aligns with the TC size which must be
1807 1806 * programmed as a power of two. See the queue mapping
1808 1807 * description in section 7.4.9.5.5.1.
1809 1808 *
1810 1809 * If i40e_intr_count - 1 is not a power of two then
1811 1810 * some queue pairs on the same VSI will have to share
1812 1811 * an interrupt.
1813 1812 *
1814 1813 * We may want to revisit this logic in a future where
1815 1814 * we have more interrupts and more VSIs. Otherwise,
1816 1815 * each VSI will use as many interrupts as possible.
1817 1816 * Using more QPs per VSI means better RSS for each
1818 1817 * group, but at the same time may require more
1819 1818 * sharing of interrupts across VSIs. This may be a
1820 1819 * good candidate for a .conf tunable.
1821 1820 */
1822 1821 n = 0x1 << ddi_fls(n);
1823 1822 i40e->i40e_num_trqpairs_per_vsi = n;
1824 1823 ASSERT3U(i40e->i40e_num_rx_groups, >, 0);
1825 1824 i40e->i40e_num_trqpairs = i40e->i40e_num_trqpairs_per_vsi *
1826 1825 i40e->i40e_num_rx_groups;
1827 1826 return (B_TRUE);
1828 1827 }
1829 1828
1830 1829 /*
1831 1830 * We only use multiple transmit/receive pairs when MSI-X interrupts are
1832 1831 * available due to the fact that the device basically only supports a
1833 1832 * single MSI interrupt.
1834 1833 */
1835 1834 i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX;
1836 1835 i40e->i40e_num_trqpairs_per_vsi = i40e->i40e_num_trqpairs;
1837 1836 i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX;
1838 1837
1839 1838 if ((intr_types & DDI_INTR_TYPE_MSI) &&
1840 1839 (i40e->i40e_intr_force <= I40E_INTR_MSI)) {
1841 1840 if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSI))
1842 1841 return (B_TRUE);
1843 1842 }
1844 1843
1845 1844 if (intr_types & DDI_INTR_TYPE_FIXED) {
1846 1845 if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_FIXED))
1847 1846 return (B_TRUE);
1848 1847 }
1849 1848
1850 1849 return (B_FALSE);
1851 1850 }
1852 1851
1853 1852 /*
1854 1853 * Map different interrupts to MSI-X vectors.
1855 1854 */
1856 1855 static boolean_t
1857 1856 i40e_map_intrs_to_vectors(i40e_t *i40e)
1858 1857 {
1859 1858 if (i40e->i40e_intr_type != DDI_INTR_TYPE_MSIX) {
1860 1859 return (B_TRUE);
1861 1860 }
1862 1861
1863 1862 /*
1864 1863 * Each queue pair is mapped to a single interrupt, so
1865 1864 * transmit and receive interrupts for a given queue share the
1866 1865 * same vector. Vector zero is reserved for the admin queue.
1867 1866 */
1868 1867 for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1869 1868 uint_t vector = i % (i40e->i40e_intr_count - 1);
1870 1869
1871 1870 i40e->i40e_trqpairs[i].itrq_rx_intrvec = vector + 1;
1872 1871 i40e->i40e_trqpairs[i].itrq_tx_intrvec = vector + 1;
1873 1872 }
1874 1873
1875 1874 return (B_TRUE);
1876 1875 }
1877 1876
1878 1877 static boolean_t
1879 1878 i40e_add_intr_handlers(i40e_t *i40e)
1880 1879 {
1881 1880 int rc, vector;
1882 1881
1883 1882 switch (i40e->i40e_intr_type) {
1884 1883 case DDI_INTR_TYPE_MSIX:
1885 1884 for (vector = 0; vector < i40e->i40e_intr_count; vector++) {
1886 1885 rc = ddi_intr_add_handler(
1887 1886 i40e->i40e_intr_handles[vector],
1888 1887 (ddi_intr_handler_t *)i40e_intr_msix, i40e,
1889 1888 (void *)(uintptr_t)vector);
1890 1889 if (rc != DDI_SUCCESS) {
1891 1890 i40e_log(i40e, "Add interrupt handler (MSI-X) "
1892 1891 "failed: return %d, vector %d", rc, vector);
1893 1892 for (vector--; vector >= 0; vector--) {
1894 1893 (void) ddi_intr_remove_handler(
1895 1894 i40e->i40e_intr_handles[vector]);
1896 1895 }
1897 1896 return (B_FALSE);
1898 1897 }
1899 1898 }
1900 1899 break;
1901 1900 case DDI_INTR_TYPE_MSI:
1902 1901 rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0],
1903 1902 (ddi_intr_handler_t *)i40e_intr_msi, i40e, NULL);
1904 1903 if (rc != DDI_SUCCESS) {
1905 1904 i40e_log(i40e, "Add interrupt handler (MSI) failed: "
1906 1905 "return %d", rc);
1907 1906 return (B_FALSE);
1908 1907 }
1909 1908 break;
1910 1909 case DDI_INTR_TYPE_FIXED:
1911 1910 rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0],
1912 1911 (ddi_intr_handler_t *)i40e_intr_legacy, i40e, NULL);
1913 1912 if (rc != DDI_SUCCESS) {
1914 1913 i40e_log(i40e, "Add interrupt handler (legacy) failed:"
1915 1914 " return %d", rc);
1916 1915 return (B_FALSE);
1917 1916 }
1918 1917 break;
1919 1918 default:
1920 1919 /* Cast to pacify lint */
1921 1920 panic("i40e_intr_type %p contains an unknown type: %d",
1922 1921 (void *)i40e, i40e->i40e_intr_type);
1923 1922 }
1924 1923
1925 1924 return (B_TRUE);
1926 1925 }
1927 1926
1928 1927 /*
1929 1928 * Perform periodic checks. Longer term, we should be thinking about additional
1930 1929 * things here:
1931 1930 *
1932 1931 * o Stall Detection
1933 1932 * o Temperature sensor detection
1934 1933 * o Device resetting
1935 1934 * o Statistics updating to avoid wraparound
1936 1935 */
1937 1936 static void
1938 1937 i40e_timer(void *arg)
1939 1938 {
1940 1939 i40e_t *i40e = arg;
1941 1940
1942 1941 mutex_enter(&i40e->i40e_general_lock);
1943 1942 i40e_link_check(i40e);
1944 1943 mutex_exit(&i40e->i40e_general_lock);
1945 1944 }
1946 1945
1947 1946 /*
1948 1947 * Get the hardware state, and scribble away anything that needs scribbling.
1949 1948 */
1950 1949 static void
1951 1950 i40e_get_hw_state(i40e_t *i40e, i40e_hw_t *hw)
1952 1951 {
1953 1952 int rc;
1954 1953
1955 1954 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
1956 1955
1957 1956 (void) i40e_aq_get_link_info(hw, TRUE, NULL, NULL);
1958 1957 i40e_link_check(i40e);
1959 1958
1960 1959 /*
1961 1960 * Try and determine our PHY. Note that we may have to retry to and
1962 1961 * delay to detect fiber correctly.
1963 1962 */
1964 1963 rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE, &i40e->i40e_phy,
1965 1964 NULL);
1966 1965 if (rc == I40E_ERR_UNKNOWN_PHY) {
1967 1966 i40e_msec_delay(200);
1968 1967 rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE,
1969 1968 &i40e->i40e_phy, NULL);
1970 1969 }
1971 1970
1972 1971 if (rc != I40E_SUCCESS) {
1973 1972 if (rc == I40E_ERR_UNKNOWN_PHY) {
1974 1973 i40e_error(i40e, "encountered unknown PHY type, "
1975 1974 "not attaching.");
1976 1975 } else {
1977 1976 i40e_error(i40e, "error getting physical capabilities: "
1978 1977 "%d, %d", rc, hw->aq.asq_last_status);
1979 1978 }
1980 1979 }
1981 1980
1982 1981 rc = i40e_update_link_info(hw);
1983 1982 if (rc != I40E_SUCCESS) {
1984 1983 i40e_error(i40e, "failed to update link information: %d", rc);
1985 1984 }
1986 1985
1987 1986 /*
1988 1987 * In general, we don't want to mask off (as in stop from being a cause)
1989 1988 * any of the interrupts that the phy might be able to generate.
1990 1989 */
1991 1990 rc = i40e_aq_set_phy_int_mask(hw, 0, NULL);
1992 1991 if (rc != I40E_SUCCESS) {
1993 1992 i40e_error(i40e, "failed to update phy link mask: %d", rc);
1994 1993 }
1995 1994 }
1996 1995
1997 1996 /*
1998 1997 * Go through and re-initialize any existing filters that we may have set up for
1999 1998 * this device. Note that we would only expect them to exist if hardware had
2000 1999 * already been initialized and we had just reset it. While we're not
2001 2000 * implementing this yet, we're keeping this around for when we add reset
2002 2001 * capabilities, so this isn't forgotten.
2003 2002 */
2004 2003 /* ARGSUSED */
2005 2004 static void
2006 2005 i40e_init_macaddrs(i40e_t *i40e, i40e_hw_t *hw)
2007 2006 {
2008 2007 }
2009 2008
2010 2009 /*
2011 2010 * Set the properties which have common values across all the VSIs.
2012 2011 * Consult the "Add VSI" command section (7.4.9.5.5.1) for a
2013 2012 * complete description of these properties.
2014 2013 */
2015 2014 static void
2016 2015 i40e_set_shared_vsi_props(i40e_t *i40e,
2017 2016 struct i40e_aqc_vsi_properties_data *info, uint_t vsi_idx)
2018 2017 {
2019 2018 uint_t tc_queues;
2020 2019 uint16_t vsi_qp_base;
2021 2020
2022 2021 /*
2023 2022 * It's important that we use bitwise-OR here; callers to this
2024 2023 * function might enable other sections before calling this
2025 2024 * function.
2026 2025 */
2027 2026 info->valid_sections |= LE_16(I40E_AQ_VSI_PROP_QUEUE_MAP_VALID |
2028 2027 I40E_AQ_VSI_PROP_VLAN_VALID);
2029 2028
2030 2029 /*
2031 2030 * Calculate the starting QP index for this VSI. This base is
2032 2031 * relative to the PF queue space; so a value of 0 for PF#1
2033 2032 * represents the absolute index PFLAN_QALLOC_FIRSTQ for PF#1.
2034 2033 */
2035 2034 vsi_qp_base = vsi_idx * i40e->i40e_num_trqpairs_per_vsi;
2036 2035 info->mapping_flags = LE_16(I40E_AQ_VSI_QUE_MAP_CONTIG);
2037 2036 info->queue_mapping[0] =
2038 2037 LE_16((vsi_qp_base << I40E_AQ_VSI_QUEUE_SHIFT) &
2039 2038 I40E_AQ_VSI_QUEUE_MASK);
2040 2039
2041 2040 /*
2042 2041 * tc_queues determines the size of the traffic class, where
2043 2042 * the size is 2^^tc_queues to a maximum of 64 for the X710
2044 2043 * and 128 for the X722.
2045 2044 *
2046 2045 * Some examples:
2047 2046 * i40e_num_trqpairs_per_vsi == 1 => tc_queues = 0, 2^^0 = 1.
2048 2047 * i40e_num_trqpairs_per_vsi == 7 => tc_queues = 3, 2^^3 = 8.
2049 2048 * i40e_num_trqpairs_per_vsi == 8 => tc_queues = 3, 2^^3 = 8.
2050 2049 * i40e_num_trqpairs_per_vsi == 9 => tc_queues = 4, 2^^4 = 16.
2051 2050 * i40e_num_trqpairs_per_vsi == 17 => tc_queues = 5, 2^^5 = 32.
2052 2051 * i40e_num_trqpairs_per_vsi == 64 => tc_queues = 6, 2^^6 = 64.
2053 2052 */
2054 2053 tc_queues = ddi_fls(i40e->i40e_num_trqpairs_per_vsi - 1);
2055 2054
2056 2055 /*
2057 2056 * The TC queue mapping is in relation to the VSI queue space.
2058 2057 * Since we are only using one traffic class (TC0) we always
2059 2058 * start at queue offset 0.
2060 2059 */
2061 2060 info->tc_mapping[0] =
2062 2061 LE_16(((0 << I40E_AQ_VSI_TC_QUE_OFFSET_SHIFT) &
2063 2062 I40E_AQ_VSI_TC_QUE_OFFSET_MASK) |
2064 2063 ((tc_queues << I40E_AQ_VSI_TC_QUE_NUMBER_SHIFT) &
2065 2064 I40E_AQ_VSI_TC_QUE_NUMBER_MASK));
2066 2065
2067 2066 /*
2068 2067 * I40E_AQ_VSI_PVLAN_MODE_ALL ("VLAN driver insertion mode")
2069 2068 *
2070 2069 * Allow tagged and untagged packets to be sent to this
2071 2070 * VSI from the host.
2072 2071 *
2073 2072 * I40E_AQ_VSI_PVLAN_EMOD_NOTHING ("VLAN and UP expose mode")
2074 2073 *
2075 2074 * Leave the tag on the frame and place no VLAN
2076 2075 * information in the descriptor. We want this mode
2077 2076 * because our MAC layer will take care of the VLAN tag,
2078 2077 * if there is one.
2079 2078 */
2080 2079 info->port_vlan_flags = I40E_AQ_VSI_PVLAN_MODE_ALL |
2081 2080 I40E_AQ_VSI_PVLAN_EMOD_NOTHING;
2082 2081 }
2083 2082
2084 2083 /*
2085 2084 * Delete the VSI at this index, if one exists. We assume there is no
2086 2085 * action we can take if this command fails but to log the failure.
2087 2086 */
2088 2087 static void
2089 2088 i40e_delete_vsi(i40e_t *i40e, uint_t idx)
2090 2089 {
2091 2090 i40e_hw_t *hw = &i40e->i40e_hw_space;
2092 2091 uint16_t seid = i40e->i40e_vsis[idx].iv_seid;
2093 2092
2094 2093 if (seid != 0) {
2095 2094 int rc;
2096 2095
2097 2096 rc = i40e_aq_delete_element(hw, seid, NULL);
2098 2097
2099 2098 if (rc != I40E_SUCCESS) {
2100 2099 i40e_error(i40e, "Failed to delete VSI %d: %d",
2101 2100 rc, hw->aq.asq_last_status);
2102 2101 }
2103 2102
2104 2103 i40e->i40e_vsis[idx].iv_seid = 0;
2105 2104 }
2106 2105 }
2107 2106
2108 2107 /*
2109 2108 * Add a new VSI.
2110 2109 */
2111 2110 static boolean_t
2112 2111 i40e_add_vsi(i40e_t *i40e, i40e_hw_t *hw, uint_t idx)
2113 2112 {
2114 2113 struct i40e_vsi_context ctx;
2115 2114 i40e_rx_group_t *rxg;
2116 2115 int rc;
2117 2116
2118 2117 /*
2119 2118 * The default VSI is created by the controller. This function
2120 2119 * creates new, non-defualt VSIs only.
2121 2120 */
2122 2121 ASSERT3U(idx, !=, 0);
2123 2122
2124 2123 bzero(&ctx, sizeof (struct i40e_vsi_context));
2125 2124 ctx.uplink_seid = i40e->i40e_veb_seid;
2126 2125 ctx.pf_num = hw->pf_id;
2127 2126 ctx.flags = I40E_AQ_VSI_TYPE_PF;
2128 2127 ctx.connection_type = I40E_AQ_VSI_CONN_TYPE_NORMAL;
2129 2128 i40e_set_shared_vsi_props(i40e, &ctx.info, idx);
2130 2129
2131 2130 rc = i40e_aq_add_vsi(hw, &ctx, NULL);
2132 2131 if (rc != I40E_SUCCESS) {
2133 2132 i40e_error(i40e, "i40e_aq_add_vsi() failed %d: %d", rc,
2134 2133 hw->aq.asq_last_status);
2135 2134 return (B_FALSE);
2136 2135 }
2137 2136
2138 2137 rxg = &i40e->i40e_rx_groups[idx];
2139 2138 rxg->irg_vsi_seid = ctx.seid;
2140 2139 i40e->i40e_vsis[idx].iv_number = ctx.vsi_number;
2141 2140 i40e->i40e_vsis[idx].iv_seid = ctx.seid;
2142 2141 i40e->i40e_vsis[idx].iv_stats_id = LE_16(ctx.info.stat_counter_idx);
2143 2142
2144 2143 if (i40e_stat_vsi_init(i40e, idx) == B_FALSE)
2145 2144 return (B_FALSE);
2146 2145
2147 2146 return (B_TRUE);
2148 2147 }
2149 2148
2150 2149 /*
2151 2150 * Configure the hardware for the Default Virtual Station Interface (VSI).
2152 2151 */
2153 2152 static boolean_t
2154 2153 i40e_config_def_vsi(i40e_t *i40e, i40e_hw_t *hw)
2155 2154 {
2156 2155 struct i40e_vsi_context ctx;
2157 2156 i40e_rx_group_t *def_rxg;
2158 2157 int err;
2159 2158 struct i40e_aqc_remove_macvlan_element_data filt;
2160 2159
2161 2160 bzero(&ctx, sizeof (struct i40e_vsi_context));
2162 2161 ctx.seid = I40E_DEF_VSI_SEID(i40e);
2163 2162 ctx.pf_num = hw->pf_id;
2164 2163 err = i40e_aq_get_vsi_params(hw, &ctx, NULL);
2165 2164 if (err != I40E_SUCCESS) {
2166 2165 i40e_error(i40e, "get VSI params failed with %d", err);
2167 2166 return (B_FALSE);
2168 2167 }
2169 2168
2170 2169 ctx.info.valid_sections = 0;
2171 2170 i40e->i40e_vsis[0].iv_number = ctx.vsi_number;
2172 2171 i40e->i40e_vsis[0].iv_stats_id = LE_16(ctx.info.stat_counter_idx);
2173 2172 if (i40e_stat_vsi_init(i40e, 0) == B_FALSE)
2174 2173 return (B_FALSE);
2175 2174
2176 2175 i40e_set_shared_vsi_props(i40e, &ctx.info, I40E_DEF_VSI_IDX);
2177 2176
2178 2177 err = i40e_aq_update_vsi_params(hw, &ctx, NULL);
2179 2178 if (err != I40E_SUCCESS) {
2180 2179 i40e_error(i40e, "Update VSI params failed with %d", err);
2181 2180 return (B_FALSE);
2182 2181 }
2183 2182
2184 2183 def_rxg = &i40e->i40e_rx_groups[0];
2185 2184 def_rxg->irg_vsi_seid = I40E_DEF_VSI_SEID(i40e);
2186 2185
2187 2186 /*
2188 2187 * We have seen three different behaviors in regards to the
2189 2188 * Default VSI and its implicit L2 MAC+VLAN filter.
2190 2189 *
2191 2190 * 1. It has an implicit filter for the factory MAC address
2192 2191 * and this filter counts against 'ifr_nmacfilt_used'.
2193 2192 *
2194 2193 * 2. It has an implicit filter for the factory MAC address
2195 2194 * and this filter DOES NOT count against 'ifr_nmacfilt_used'.
2196 2195 *
2197 2196 * 3. It DOES NOT have an implicit filter.
2198 2197 *
2199 2198 * All three of these cases are accounted for below. If we
2200 2199 * fail to remove the L2 filter (ENOENT) then we assume there
2201 2200 * wasn't one. Otherwise, if we successfully remove the
2202 2201 * filter, we make sure to update the 'ifr_nmacfilt_used'
2203 2202 * count accordingly.
2204 2203 *
2205 2204 * We remove this filter to prevent duplicate delivery of
2206 2205 * packets destined for the primary MAC address as DLS will
2207 2206 * create the same filter on a non-default VSI for the primary
2208 2207 * MAC client.
2209 2208 *
2210 2209 * If you change the following code please test it across as
2211 2210 * many X700 series controllers and firmware revisions as you
2212 2211 * can.
2213 2212 */
2214 2213 bzero(&filt, sizeof (filt));
2215 2214 bcopy(hw->mac.port_addr, filt.mac_addr, ETHERADDRL);
2216 2215 filt.flags = I40E_AQC_MACVLAN_DEL_PERFECT_MATCH;
2217 2216 filt.vlan_tag = 0;
2218 2217
2219 2218 ASSERT3U(i40e->i40e_resources.ifr_nmacfilt_used, <=, 1);
2220 2219 i40e_log(i40e, "Num L2 filters: %u",
2221 2220 i40e->i40e_resources.ifr_nmacfilt_used);
2222 2221
2223 2222 err = i40e_aq_remove_macvlan(hw, I40E_DEF_VSI_SEID(i40e), &filt, 1,
2224 2223 NULL);
2225 2224 if (err == I40E_SUCCESS) {
2226 2225 i40e_log(i40e,
2227 2226 "Removed L2 filter from Default VSI with SEID %u",
2228 2227 I40E_DEF_VSI_SEID(i40e));
2229 2228 } else if (hw->aq.asq_last_status == ENOENT) {
2230 2229 i40e_log(i40e,
2231 2230 "No L2 filter for Default VSI with SEID %u",
2232 2231 I40E_DEF_VSI_SEID(i40e));
2233 2232 } else {
2234 2233 i40e_error(i40e, "Failed to remove L2 filter from"
2235 2234 " Default VSI with SEID %u: %d (%d)",
2236 2235 I40E_DEF_VSI_SEID(i40e), err, hw->aq.asq_last_status);
2237 2236
2238 2237 return (B_FALSE);
2239 2238 }
2240 2239
2241 2240 /*
2242 2241 * As mentioned above, the controller created an implicit L2
2243 2242 * filter for the primary MAC. We want to remove both the
2244 2243 * filter and decrement the filter count. However, not all
2245 2244 * controllers count this implicit filter against the total
2246 2245 * MAC filter count. So here we are making sure it is either
2247 2246 * one or zero. If it is one, then we know it is for the
2248 2247 * implicit filter and we should decrement since we just
2249 2248 * removed the filter above. If it is zero then we know the
2250 2249 * controller that does not count the implicit filter, and it
2251 2250 * was enough to just remove it; we leave the count alone.
2252 2251 * But if it is neither, then we have never seen a controller
2253 2252 * like this before and we should fail to attach.
2254 2253 *
2255 2254 * It is unfortunate that this code must exist but the
2256 2255 * behavior of this implicit L2 filter and its corresponding
2257 2256 * count were dicovered through empirical testing. The
2258 2257 * programming manuals hint at this filter but do not
2259 2258 * explicitly call out the exact behavior.
2260 2259 */
2261 2260 if (i40e->i40e_resources.ifr_nmacfilt_used == 1) {
2262 2261 i40e->i40e_resources.ifr_nmacfilt_used--;
2263 2262 } else {
2264 2263 if (i40e->i40e_resources.ifr_nmacfilt_used != 0) {
2265 2264 i40e_error(i40e, "Unexpected L2 filter count: %u"
2266 2265 " (expected 0)",
2267 2266 i40e->i40e_resources.ifr_nmacfilt_used);
2268 2267 return (B_FALSE);
2269 2268 }
2270 2269 }
2271 2270
2272 2271 return (B_TRUE);
2273 2272 }
2274 2273
2275 2274 static boolean_t
2276 2275 i40e_config_rss_key_x722(i40e_t *i40e, i40e_hw_t *hw)
2277 2276 {
2278 2277 for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) {
2279 2278 uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1];
2280 2279 struct i40e_aqc_get_set_rss_key_data key;
2281 2280 const char *u8seed;
2282 2281 enum i40e_status_code status;
2283 2282 uint16_t vsi_number = i40e->i40e_vsis[i].iv_number;
2284 2283
2285 2284 (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed));
2286 2285 u8seed = (char *)seed;
2287 2286
2288 2287 CTASSERT(sizeof (key) >= (sizeof (key.standard_rss_key) +
2289 2288 sizeof (key.extended_hash_key)));
2290 2289
2291 2290 bcopy(u8seed, key.standard_rss_key,
2292 2291 sizeof (key.standard_rss_key));
2293 2292 bcopy(&u8seed[sizeof (key.standard_rss_key)],
2294 2293 key.extended_hash_key, sizeof (key.extended_hash_key));
2295 2294
2296 2295 ASSERT3U(vsi_number, !=, 0);
2297 2296 status = i40e_aq_set_rss_key(hw, vsi_number, &key);
2298 2297
2299 2298 if (status != I40E_SUCCESS) {
2300 2299 i40e_error(i40e, "failed to set RSS key for VSI %u: %d",
2301 2300 vsi_number, status);
2302 2301 return (B_FALSE);
2303 2302 }
2304 2303 }
2305 2304
2306 2305 return (B_TRUE);
2307 2306 }
2308 2307
2309 2308 /*
2310 2309 * Configure the RSS key. For the X710 controller family, this is set on a
2311 2310 * per-PF basis via registers. For the X722, this is done on a per-VSI basis
2312 2311 * through the admin queue.
2313 2312 */
2314 2313 static boolean_t
2315 2314 i40e_config_rss_key(i40e_t *i40e, i40e_hw_t *hw)
2316 2315 {
2317 2316 if (i40e_is_x722(i40e)) {
2318 2317 if (!i40e_config_rss_key_x722(i40e, hw))
2319 2318 return (B_FALSE);
2320 2319 } else {
2321 2320 uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1];
2322 2321
2323 2322 (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed));
2324 2323 for (uint_t i = 0; i <= I40E_PFQF_HKEY_MAX_INDEX; i++)
2325 2324 i40e_write_rx_ctl(hw, I40E_PFQF_HKEY(i), seed[i]);
2326 2325 }
2327 2326
2328 2327 return (B_TRUE);
2329 2328 }
2330 2329
2331 2330 /*
2332 2331 * Populate the LUT. The size of each entry in the LUT depends on the controller
2333 2332 * family, with the X722 using a known 7-bit width. On the X710 controller, this
2334 2333 * is programmed through its control registers where as on the X722 this is
2335 2334 * configured through the admin queue. Also of note, the X722 allows the LUT to
2336 2335 * be set on a per-PF or VSI basis. At this time we use the PF setting. If we
2337 2336 * decide to use the per-VSI LUT in the future, then we will need to modify the
2338 2337 * i40e_add_vsi() function to set the RSS LUT bits in the queueing section.
2339 2338 *
2340 2339 * We populate the LUT in a round robin fashion with the rx queue indices from 0
2341 2340 * to i40e_num_trqpairs_per_vsi - 1.
2342 2341 */
2343 2342 static boolean_t
2344 2343 i40e_config_rss_hlut(i40e_t *i40e, i40e_hw_t *hw)
2345 2344 {
2346 2345 uint32_t *hlut;
2347 2346 uint8_t lut_mask;
2348 2347 uint_t i;
2349 2348 boolean_t ret = B_FALSE;
2350 2349
2351 2350 /*
2352 2351 * We always configure the PF with a table size of 512 bytes in
2353 2352 * i40e_chip_start().
2354 2353 */
2355 2354 hlut = kmem_alloc(I40E_HLUT_TABLE_SIZE, KM_NOSLEEP);
2356 2355 if (hlut == NULL) {
2357 2356 i40e_error(i40e, "i40e_config_rss() buffer allocation failed");
2358 2357 return (B_FALSE);
2359 2358 }
2360 2359
2361 2360 /*
2362 2361 * The width of the X722 is apparently defined to be 7 bits, regardless
2363 2362 * of the capability.
2364 2363 */
2365 2364 if (i40e_is_x722(i40e)) {
2366 2365 lut_mask = (1 << 7) - 1;
2367 2366 } else {
2368 2367 lut_mask = (1 << hw->func_caps.rss_table_entry_width) - 1;
2369 2368 }
2370 2369
2371 2370 for (i = 0; i < I40E_HLUT_TABLE_SIZE; i++) {
2372 2371 ((uint8_t *)hlut)[i] =
2373 2372 (i % i40e->i40e_num_trqpairs_per_vsi) & lut_mask;
2374 2373 }
2375 2374
2376 2375 if (i40e_is_x722(i40e)) {
2377 2376 enum i40e_status_code status;
2378 2377
2379 2378 status = i40e_aq_set_rss_lut(hw, 0, B_TRUE, (uint8_t *)hlut,
2380 2379 I40E_HLUT_TABLE_SIZE);
2381 2380
2382 2381 if (status != I40E_SUCCESS) {
2383 2382 i40e_error(i40e, "failed to set RSS LUT %d: %d",
2384 2383 status, hw->aq.asq_last_status);
2385 2384 goto out;
2386 2385 }
2387 2386 } else {
2388 2387 for (i = 0; i < I40E_HLUT_TABLE_SIZE >> 2; i++) {
2389 2388 I40E_WRITE_REG(hw, I40E_PFQF_HLUT(i), hlut[i]);
2390 2389 }
2391 2390 }
2392 2391 ret = B_TRUE;
2393 2392 out:
2394 2393 kmem_free(hlut, I40E_HLUT_TABLE_SIZE);
2395 2394 return (ret);
2396 2395 }
2397 2396
2398 2397 /*
2399 2398 * Set up RSS.
2400 2399 * 1. Seed the hash key.
2401 2400 * 2. Enable PCTYPEs for the hash filter.
2402 2401 * 3. Populate the LUT.
2403 2402 */
2404 2403 static boolean_t
2405 2404 i40e_config_rss(i40e_t *i40e, i40e_hw_t *hw)
2406 2405 {
2407 2406 uint64_t hena;
2408 2407
2409 2408 /*
2410 2409 * 1. Seed the hash key
2411 2410 */
2412 2411 if (!i40e_config_rss_key(i40e, hw))
2413 2412 return (B_FALSE);
2414 2413
2415 2414 /*
2416 2415 * 2. Configure PCTYPES
2417 2416 */
2418 2417 hena = (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) |
2419 2418 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP) |
2420 2419 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_SCTP) |
2421 2420 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_UDP) |
2422 2421 (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV4) |
2423 2422 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_OTHER) |
2424 2423 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP) |
2425 2424 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_SCTP) |
2426 2425 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_UDP) |
2427 2426 (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV6) |
2428 2427 (1ULL << I40E_FILTER_PCTYPE_L2_PAYLOAD);
2429 2428
2430 2429 /*
2431 2430 * Add additional types supported by the X722 controller.
2432 2431 */
2433 2432 if (i40e_is_x722(i40e)) {
2434 2433 hena |= (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV4_UDP) |
2435 2434 (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV4_UDP) |
2436 2435 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP_SYN_NO_ACK) |
2437 2436 (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV6_UDP) |
2438 2437 (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV6_UDP) |
2439 2438 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP_SYN_NO_ACK);
2440 2439 }
2441 2440
2442 2441 i40e_write_rx_ctl(hw, I40E_PFQF_HENA(0), (uint32_t)hena);
2443 2442 i40e_write_rx_ctl(hw, I40E_PFQF_HENA(1), (uint32_t)(hena >> 32));
2444 2443
2445 2444 /*
2446 2445 * 3. Populate LUT
2447 2446 */
2448 2447 return (i40e_config_rss_hlut(i40e, hw));
2449 2448 }
2450 2449
2451 2450 /*
2452 2451 * Wrapper to kick the chipset on.
2453 2452 */
2454 2453 static boolean_t
2455 2454 i40e_chip_start(i40e_t *i40e)
2456 2455 {
2457 2456 i40e_hw_t *hw = &i40e->i40e_hw_space;
2458 2457 struct i40e_filter_control_settings filter;
2459 2458 int rc;
2460 2459 uint8_t err;
2461 2460
2462 2461 if (((hw->aq.fw_maj_ver == 4) && (hw->aq.fw_min_ver < 33)) ||
2463 2462 (hw->aq.fw_maj_ver < 4)) {
2464 2463 i40e_msec_delay(75);
2465 2464 if (i40e_aq_set_link_restart_an(hw, TRUE, NULL) !=
2466 2465 I40E_SUCCESS) {
2467 2466 i40e_error(i40e, "failed to restart link: admin queue "
2468 2467 "error: %d", hw->aq.asq_last_status);
2469 2468 return (B_FALSE);
2470 2469 }
2471 2470 }
2472 2471
2473 2472 /* Determine hardware state */
2474 2473 i40e_get_hw_state(i40e, hw);
2475 2474
2476 2475 /* For now, we always disable Ethernet Flow Control. */
2477 2476 hw->fc.requested_mode = I40E_FC_NONE;
2478 2477 rc = i40e_set_fc(hw, &err, B_TRUE);
2479 2478 if (rc != I40E_SUCCESS) {
2480 2479 i40e_error(i40e, "Setting flow control failed, returned %d"
2481 2480 " with error: 0x%x", rc, err);
2482 2481 return (B_FALSE);
2483 2482 }
2484 2483
2485 2484 /* Initialize mac addresses. */
2486 2485 i40e_init_macaddrs(i40e, hw);
2487 2486
2488 2487 /*
2489 2488 * Set up the filter control. If the hash lut size is changed from
2490 2489 * I40E_HASH_LUT_SIZE_512 then I40E_HLUT_TABLE_SIZE and
2491 2490 * i40e_config_rss_hlut() will need to be updated.
2492 2491 */
2493 2492 bzero(&filter, sizeof (filter));
2494 2493 filter.enable_ethtype = TRUE;
2495 2494 filter.enable_macvlan = TRUE;
2496 2495 filter.hash_lut_size = I40E_HASH_LUT_SIZE_512;
2497 2496
2498 2497 rc = i40e_set_filter_control(hw, &filter);
2499 2498 if (rc != I40E_SUCCESS) {
2500 2499 i40e_error(i40e, "i40e_set_filter_control() returned %d", rc);
2501 2500 return (B_FALSE);
2502 2501 }
2503 2502
2504 2503 i40e_intr_chip_init(i40e);
2505 2504
2506 2505 rc = i40e_get_mac_seid(i40e);
2507 2506 if (rc == -1) {
2508 2507 i40e_error(i40e, "failed to obtain MAC Uplink SEID");
2509 2508 return (B_FALSE);
2510 2509 }
2511 2510 i40e->i40e_mac_seid = (uint16_t)rc;
2512 2511
2513 2512 /*
2514 2513 * Create a VEB in order to support multiple VSIs. Each VSI
2515 2514 * functions as a MAC group. This call sets the PF's MAC as
2516 2515 * the uplink port and the PF's default VSI as the default
2517 2516 * downlink port.
2518 2517 */
2519 2518 rc = i40e_aq_add_veb(hw, i40e->i40e_mac_seid, I40E_DEF_VSI_SEID(i40e),
2520 2519 0x1, B_TRUE, &i40e->i40e_veb_seid, B_FALSE, NULL);
2521 2520 if (rc != I40E_SUCCESS) {
2522 2521 i40e_error(i40e, "i40e_aq_add_veb() failed %d: %d", rc,
2523 2522 hw->aq.asq_last_status);
2524 2523 return (B_FALSE);
2525 2524 }
2526 2525
2527 2526 if (!i40e_config_def_vsi(i40e, hw))
2528 2527 return (B_FALSE);
2529 2528
2530 2529 for (uint_t i = 1; i < i40e->i40e_num_rx_groups; i++) {
2531 2530 if (!i40e_add_vsi(i40e, hw, i))
2532 2531 return (B_FALSE);
2533 2532 }
2534 2533
2535 2534 if (!i40e_config_rss(i40e, hw))
2536 2535 return (B_FALSE);
2537 2536
2538 2537 i40e_flush(hw);
2539 2538
2540 2539 return (B_TRUE);
2541 2540 }
2542 2541
2543 2542 /*
2544 2543 * Take care of tearing down the rx ring. See 8.3.3.1.2 for more information.
2545 2544 */
2546 2545 static void
2547 2546 i40e_shutdown_rx_rings(i40e_t *i40e)
2548 2547 {
2549 2548 int i;
2550 2549 uint32_t reg;
2551 2550
2552 2551 i40e_hw_t *hw = &i40e->i40e_hw_space;
2553 2552
2554 2553 /*
2555 2554 * Step 1. The interrupt linked list (see i40e_intr.c for more
2556 2555 * information) should have already been cleared before calling this
2557 2556 * function.
2558 2557 */
2559 2558 #ifdef DEBUG
2560 2559 if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) {
2561 2560 for (i = 1; i < i40e->i40e_intr_count; i++) {
2562 2561 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1));
2563 2562 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2564 2563 }
2565 2564 } else {
2566 2565 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0);
2567 2566 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2568 2567 }
2569 2568
2570 2569 #endif /* DEBUG */
2571 2570
2572 2571 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2573 2572 /*
2574 2573 * Step 1. Request the queue by clearing QENA_REQ. It may not be
2575 2574 * set due to unwinding from failures and a partially enabled
2576 2575 * ring set.
2577 2576 */
2578 2577 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2579 2578 if (!(reg & I40E_QRX_ENA_QENA_REQ_MASK))
2580 2579 continue;
2581 2580 VERIFY((reg & I40E_QRX_ENA_QENA_REQ_MASK) ==
2582 2581 I40E_QRX_ENA_QENA_REQ_MASK);
2583 2582 reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
2584 2583 I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg);
2585 2584 }
2586 2585
2587 2586 /*
2588 2587 * Step 2. Wait for the disable to take, by having QENA_STAT in the FPM
2589 2588 * be cleared. Note that we could still receive data in the queue during
2590 2589 * this time. We don't actually wait for this now and instead defer this
2591 2590 * to i40e_shutdown_rings_wait(), after we've interleaved disabling the
2592 2591 * TX queues as well.
2593 2592 */
2594 2593 }
2595 2594
2596 2595 static void
2597 2596 i40e_shutdown_tx_rings(i40e_t *i40e)
2598 2597 {
2599 2598 int i;
2600 2599 uint32_t reg;
2601 2600
2602 2601 i40e_hw_t *hw = &i40e->i40e_hw_space;
2603 2602
2604 2603 /*
2605 2604 * Step 1. The interrupt linked list should already have been cleared.
2606 2605 */
2607 2606 #ifdef DEBUG
2608 2607 if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) {
2609 2608 for (i = 1; i < i40e->i40e_intr_count; i++) {
2610 2609 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1));
2611 2610 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2612 2611 }
2613 2612 } else {
2614 2613 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0);
2615 2614 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2616 2615
2617 2616 }
2618 2617 #endif /* DEBUG */
2619 2618
2620 2619 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2621 2620 /*
2622 2621 * Step 2. Set the SET_QDIS flag for every queue.
2623 2622 */
2624 2623 i40e_pre_tx_queue_cfg(hw, i, B_FALSE);
2625 2624 }
2626 2625
2627 2626 /*
2628 2627 * Step 3. Wait at least 400 usec (can be done once for all queues).
2629 2628 */
2630 2629 drv_usecwait(500);
2631 2630
2632 2631 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2633 2632 /*
2634 2633 * Step 4. Clear the QENA_REQ flag which tells hardware to
2635 2634 * quiesce. If QENA_REQ is not already set then that means that
2636 2635 * we likely already tried to disable this queue.
2637 2636 */
2638 2637 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2639 2638 if (!(reg & I40E_QTX_ENA_QENA_REQ_MASK))
2640 2639 continue;
2641 2640 reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
2642 2641 I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg);
2643 2642 }
2644 2643
2645 2644 /*
2646 2645 * Step 5. Wait for all drains to finish. This will be done by the
2647 2646 * hardware removing the QENA_STAT flag from the queue. Rather than
2648 2647 * waiting here, we interleave it with all the others in
2649 2648 * i40e_shutdown_rings_wait().
2650 2649 */
2651 2650 }
2652 2651
2653 2652 /*
2654 2653 * Wait for all the rings to be shut down. e.g. Steps 2 and 5 from the above
2655 2654 * functions.
2656 2655 */
2657 2656 static boolean_t
2658 2657 i40e_shutdown_rings_wait(i40e_t *i40e)
2659 2658 {
2660 2659 int i, try;
2661 2660 i40e_hw_t *hw = &i40e->i40e_hw_space;
2662 2661
2663 2662 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2664 2663 uint32_t reg;
2665 2664
2666 2665 for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) {
2667 2666 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2668 2667 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0)
2669 2668 break;
2670 2669 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2671 2670 }
2672 2671
2673 2672 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) != 0) {
2674 2673 i40e_error(i40e, "timed out disabling rx queue %d",
2675 2674 i);
2676 2675 return (B_FALSE);
2677 2676 }
2678 2677
2679 2678 for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) {
2680 2679 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2681 2680 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0)
2682 2681 break;
2683 2682 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2684 2683 }
2685 2684
2686 2685 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) != 0) {
2687 2686 i40e_error(i40e, "timed out disabling tx queue %d",
2688 2687 i);
2689 2688 return (B_FALSE);
2690 2689 }
2691 2690 }
2692 2691
2693 2692 return (B_TRUE);
2694 2693 }
2695 2694
2696 2695 static boolean_t
2697 2696 i40e_shutdown_rings(i40e_t *i40e)
2698 2697 {
2699 2698 i40e_shutdown_rx_rings(i40e);
2700 2699 i40e_shutdown_tx_rings(i40e);
2701 2700 return (i40e_shutdown_rings_wait(i40e));
2702 2701 }
2703 2702
2704 2703 static void
2705 2704 i40e_setup_rx_descs(i40e_trqpair_t *itrq)
2706 2705 {
2707 2706 int i;
2708 2707 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2709 2708
2710 2709 for (i = 0; i < rxd->rxd_ring_size; i++) {
2711 2710 i40e_rx_control_block_t *rcb;
2712 2711 i40e_rx_desc_t *rdesc;
2713 2712
2714 2713 rcb = rxd->rxd_work_list[i];
2715 2714 rdesc = &rxd->rxd_desc_ring[i];
2716 2715
2717 2716 rdesc->read.pkt_addr =
2718 2717 CPU_TO_LE64((uintptr_t)rcb->rcb_dma.dmab_dma_address);
2719 2718 rdesc->read.hdr_addr = 0;
2720 2719 }
2721 2720 }
2722 2721
2723 2722 static boolean_t
2724 2723 i40e_setup_rx_hmc(i40e_trqpair_t *itrq)
2725 2724 {
2726 2725 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2727 2726 i40e_t *i40e = itrq->itrq_i40e;
2728 2727 i40e_hw_t *hw = &i40e->i40e_hw_space;
2729 2728
2730 2729 struct i40e_hmc_obj_rxq rctx;
2731 2730 int err;
2732 2731
2733 2732 bzero(&rctx, sizeof (struct i40e_hmc_obj_rxq));
2734 2733 rctx.base = rxd->rxd_desc_area.dmab_dma_address /
2735 2734 I40E_HMC_RX_CTX_UNIT;
2736 2735 rctx.qlen = rxd->rxd_ring_size;
2737 2736 VERIFY(i40e->i40e_rx_buf_size >= I40E_HMC_RX_DBUFF_MIN);
2738 2737 VERIFY(i40e->i40e_rx_buf_size <= I40E_HMC_RX_DBUFF_MAX);
2739 2738 rctx.dbuff = i40e->i40e_rx_buf_size >> I40E_RXQ_CTX_DBUFF_SHIFT;
2740 2739 rctx.hbuff = 0 >> I40E_RXQ_CTX_HBUFF_SHIFT;
2741 2740 rctx.dtype = I40E_HMC_RX_DTYPE_NOSPLIT;
2742 2741 rctx.dsize = I40E_HMC_RX_DSIZE_32BYTE;
2743 2742 rctx.crcstrip = I40E_HMC_RX_CRCSTRIP_ENABLE;
2744 2743 rctx.fc_ena = I40E_HMC_RX_FC_DISABLE;
2745 2744 rctx.l2tsel = I40E_HMC_RX_L2TAGORDER;
2746 2745 rctx.hsplit_0 = I40E_HMC_RX_HDRSPLIT_DISABLE;
2747 2746 rctx.hsplit_1 = I40E_HMC_RX_HDRSPLIT_DISABLE;
2748 2747 rctx.showiv = I40E_HMC_RX_INVLAN_DONTSTRIP;
2749 2748 rctx.rxmax = i40e->i40e_frame_max;
2750 2749 rctx.tphrdesc_ena = I40E_HMC_RX_TPH_DISABLE;
2751 2750 rctx.tphwdesc_ena = I40E_HMC_RX_TPH_DISABLE;
2752 2751 rctx.tphdata_ena = I40E_HMC_RX_TPH_DISABLE;
2753 2752 rctx.tphhead_ena = I40E_HMC_RX_TPH_DISABLE;
2754 2753 rctx.lrxqthresh = I40E_HMC_RX_LOWRXQ_NOINTR;
2755 2754
2756 2755 /*
2757 2756 * This must be set to 0x1, see Table 8-12 in section 8.3.3.2.2.
2758 2757 */
2759 2758 rctx.prefena = I40E_HMC_RX_PREFENA;
2760 2759
2761 2760 err = i40e_clear_lan_rx_queue_context(hw, itrq->itrq_index);
2762 2761 if (err != I40E_SUCCESS) {
2763 2762 i40e_error(i40e, "failed to clear rx queue %d context: %d",
2764 2763 itrq->itrq_index, err);
2765 2764 return (B_FALSE);
2766 2765 }
2767 2766
2768 2767 err = i40e_set_lan_rx_queue_context(hw, itrq->itrq_index, &rctx);
2769 2768 if (err != I40E_SUCCESS) {
2770 2769 i40e_error(i40e, "failed to set rx queue %d context: %d",
2771 2770 itrq->itrq_index, err);
2772 2771 return (B_FALSE);
2773 2772 }
2774 2773
2775 2774 return (B_TRUE);
2776 2775 }
2777 2776
2778 2777 /*
2779 2778 * Take care of setting up the descriptor rings and actually programming the
2780 2779 * device. See 8.3.3.1.1 for the full list of steps we need to do to enable the
2781 2780 * rx rings.
2782 2781 */
2783 2782 static boolean_t
2784 2783 i40e_setup_rx_rings(i40e_t *i40e)
2785 2784 {
2786 2785 int i;
2787 2786 i40e_hw_t *hw = &i40e->i40e_hw_space;
2788 2787
2789 2788 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2790 2789 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
2791 2790 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2792 2791 uint32_t reg;
2793 2792
2794 2793 /*
2795 2794 * Step 1. Program all receive ring descriptors.
2796 2795 */
2797 2796 i40e_setup_rx_descs(itrq);
2798 2797
2799 2798 /*
2800 2799 * Step 2. Program the queue's FPM/HMC context.
2801 2800 */
2802 2801 if (i40e_setup_rx_hmc(itrq) == B_FALSE)
2803 2802 return (B_FALSE);
2804 2803
2805 2804 /*
2806 2805 * Step 3. Clear the queue's tail pointer and set it to the end
2807 2806 * of the space.
2808 2807 */
2809 2808 I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), 0);
2810 2809 I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), rxd->rxd_ring_size - 1);
2811 2810
2812 2811 /*
2813 2812 * Step 4. Enable the queue via the QENA_REQ.
2814 2813 */
2815 2814 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2816 2815 VERIFY0(reg & (I40E_QRX_ENA_QENA_REQ_MASK |
2817 2816 I40E_QRX_ENA_QENA_STAT_MASK));
2818 2817 reg |= I40E_QRX_ENA_QENA_REQ_MASK;
2819 2818 I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg);
2820 2819 }
2821 2820
2822 2821 /*
2823 2822 * Note, we wait for every queue to be enabled before we start checking.
2824 2823 * This will hopefully cause most queues to be enabled at this point.
2825 2824 */
2826 2825 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2827 2826 uint32_t j, reg;
2828 2827
2829 2828 /*
2830 2829 * Step 5. Verify that QENA_STAT has been set. It's promised
2831 2830 * that this should occur within about 10 us, but like other
2832 2831 * systems, we give the card a bit more time.
2833 2832 */
2834 2833 for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) {
2835 2834 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2836 2835
2837 2836 if (reg & I40E_QRX_ENA_QENA_STAT_MASK)
2838 2837 break;
2839 2838 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2840 2839 }
2841 2840
2842 2841 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0) {
2843 2842 i40e_error(i40e, "failed to enable rx queue %d, timed "
2844 2843 "out.", i);
2845 2844 return (B_FALSE);
2846 2845 }
2847 2846 }
2848 2847
2849 2848 return (B_TRUE);
2850 2849 }
2851 2850
2852 2851 static boolean_t
2853 2852 i40e_setup_tx_hmc(i40e_trqpair_t *itrq)
2854 2853 {
2855 2854 i40e_t *i40e = itrq->itrq_i40e;
2856 2855 i40e_hw_t *hw = &i40e->i40e_hw_space;
2857 2856
2858 2857 struct i40e_hmc_obj_txq tctx;
2859 2858 struct i40e_vsi_context context;
2860 2859 int err;
2861 2860
2862 2861 bzero(&tctx, sizeof (struct i40e_hmc_obj_txq));
2863 2862 tctx.new_context = I40E_HMC_TX_NEW_CONTEXT;
2864 2863 tctx.base = itrq->itrq_desc_area.dmab_dma_address /
2865 2864 I40E_HMC_TX_CTX_UNIT;
2866 2865 tctx.fc_ena = I40E_HMC_TX_FC_DISABLE;
2867 2866 tctx.timesync_ena = I40E_HMC_TX_TS_DISABLE;
2868 2867 tctx.fd_ena = I40E_HMC_TX_FD_DISABLE;
2869 2868 tctx.alt_vlan_ena = I40E_HMC_TX_ALT_VLAN_DISABLE;
2870 2869 tctx.head_wb_ena = I40E_HMC_TX_WB_ENABLE;
2871 2870 tctx.qlen = itrq->itrq_tx_ring_size;
2872 2871 tctx.tphrdesc_ena = I40E_HMC_TX_TPH_DISABLE;
2873 2872 tctx.tphrpacket_ena = I40E_HMC_TX_TPH_DISABLE;
2874 2873 tctx.tphwdesc_ena = I40E_HMC_TX_TPH_DISABLE;
2875 2874 tctx.head_wb_addr = itrq->itrq_desc_area.dmab_dma_address +
2876 2875 sizeof (i40e_tx_desc_t) * itrq->itrq_tx_ring_size;
2877 2876
2878 2877 /*
2879 2878 * This field isn't actually documented, like crc, but it suggests that
2880 2879 * it should be zeroed. We leave both of these here because of that for
2881 2880 * now. We should check with Intel on why these are here even.
2882 2881 */
2883 2882 tctx.crc = 0;
2884 2883 tctx.rdylist_act = 0;
2885 2884
2886 2885 /*
2887 2886 * We're supposed to assign the rdylist field with the value of the
2888 2887 * traffic class index for the first device. We query the VSI parameters
2889 2888 * again to get what the handle is. Note that every queue is always
2890 2889 * assigned to traffic class zero, because we don't actually use them.
2891 2890 */
2892 2891 bzero(&context, sizeof (struct i40e_vsi_context));
2893 2892 context.seid = I40E_DEF_VSI_SEID(i40e);
2894 2893 context.pf_num = hw->pf_id;
2895 2894 err = i40e_aq_get_vsi_params(hw, &context, NULL);
2896 2895 if (err != I40E_SUCCESS) {
2897 2896 i40e_error(i40e, "get VSI params failed with %d", err);
2898 2897 return (B_FALSE);
2899 2898 }
2900 2899 tctx.rdylist = LE_16(context.info.qs_handle[0]);
2901 2900
2902 2901 err = i40e_clear_lan_tx_queue_context(hw, itrq->itrq_index);
2903 2902 if (err != I40E_SUCCESS) {
2904 2903 i40e_error(i40e, "failed to clear tx queue %d context: %d",
2905 2904 itrq->itrq_index, err);
2906 2905 return (B_FALSE);
2907 2906 }
2908 2907
2909 2908 err = i40e_set_lan_tx_queue_context(hw, itrq->itrq_index, &tctx);
2910 2909 if (err != I40E_SUCCESS) {
2911 2910 i40e_error(i40e, "failed to set tx queue %d context: %d",
2912 2911 itrq->itrq_index, err);
2913 2912 return (B_FALSE);
2914 2913 }
2915 2914
2916 2915 return (B_TRUE);
2917 2916 }
2918 2917
2919 2918 /*
2920 2919 * Take care of setting up the descriptor rings and actually programming the
2921 2920 * device. See 8.4.3.1.1 for what we need to do here.
2922 2921 */
2923 2922 static boolean_t
2924 2923 i40e_setup_tx_rings(i40e_t *i40e)
2925 2924 {
2926 2925 int i;
2927 2926 i40e_hw_t *hw = &i40e->i40e_hw_space;
2928 2927
2929 2928 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2930 2929 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
2931 2930 uint32_t reg;
2932 2931
2933 2932 /*
2934 2933 * Step 1. Clear the queue disable flag and verify that the
2935 2934 * index is set correctly.
2936 2935 */
2937 2936 i40e_pre_tx_queue_cfg(hw, i, B_TRUE);
2938 2937
2939 2938 /*
2940 2939 * Step 2. Prepare the queue's FPM/HMC context.
2941 2940 */
2942 2941 if (i40e_setup_tx_hmc(itrq) == B_FALSE)
2943 2942 return (B_FALSE);
2944 2943
2945 2944 /*
2946 2945 * Step 3. Verify that it's clear that this PF owns this queue.
2947 2946 */
2948 2947 reg = I40E_QTX_CTL_PF_QUEUE;
2949 2948 reg |= (hw->pf_id << I40E_QTX_CTL_PF_INDX_SHIFT) &
2950 2949 I40E_QTX_CTL_PF_INDX_MASK;
2951 2950 I40E_WRITE_REG(hw, I40E_QTX_CTL(itrq->itrq_index), reg);
2952 2951 i40e_flush(hw);
2953 2952
2954 2953 /*
2955 2954 * Step 4. Set the QENA_REQ flag.
2956 2955 */
2957 2956 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2958 2957 VERIFY0(reg & (I40E_QTX_ENA_QENA_REQ_MASK |
2959 2958 I40E_QTX_ENA_QENA_STAT_MASK));
2960 2959 reg |= I40E_QTX_ENA_QENA_REQ_MASK;
2961 2960 I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg);
2962 2961 }
2963 2962
2964 2963 /*
2965 2964 * Note, we wait for every queue to be enabled before we start checking.
2966 2965 * This will hopefully cause most queues to be enabled at this point.
2967 2966 */
2968 2967 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2969 2968 uint32_t j, reg;
2970 2969
2971 2970 /*
2972 2971 * Step 5. Verify that QENA_STAT has been set. It's promised
2973 2972 * that this should occur within about 10 us, but like BSD,
2974 2973 * we'll try for up to 100 ms for this queue.
2975 2974 */
2976 2975 for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) {
2977 2976 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2978 2977
2979 2978 if (reg & I40E_QTX_ENA_QENA_STAT_MASK)
2980 2979 break;
2981 2980 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2982 2981 }
2983 2982
2984 2983 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0) {
2985 2984 i40e_error(i40e, "failed to enable tx queue %d, timed "
2986 2985 "out", i);
2987 2986 return (B_FALSE);
2988 2987 }
2989 2988 }
2990 2989
2991 2990 return (B_TRUE);
2992 2991 }
2993 2992
2994 2993 void
2995 2994 i40e_stop(i40e_t *i40e, boolean_t free_allocations)
2996 2995 {
2997 2996 uint_t i;
2998 2997 i40e_hw_t *hw = &i40e->i40e_hw_space;
2999 2998
3000 2999 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
3001 3000
3002 3001 /*
3003 3002 * Shutdown and drain the tx and rx pipeline. We do this using the
3004 3003 * following steps.
3005 3004 *
3006 3005 * 1) Shutdown interrupts to all the queues (trying to keep the admin
3007 3006 * queue alive).
3008 3007 *
3009 3008 * 2) Remove all of the interrupt tx and rx causes by setting the
3010 3009 * interrupt linked lists to zero.
3011 3010 *
3012 3011 * 2) Shutdown the tx and rx rings. Because i40e_shutdown_rings() should
3013 3012 * wait for all the queues to be disabled, once we reach that point
3014 3013 * it should be safe to free associated data.
3015 3014 *
3016 3015 * 4) Wait 50ms after all that is done. This ensures that the rings are
3017 3016 * ready for programming again and we don't have to think about this
3018 3017 * in other parts of the driver.
3019 3018 *
3020 3019 * 5) Disable remaining chip interrupts, (admin queue, etc.)
3021 3020 *
3022 3021 * 6) Verify that FM is happy with all the register accesses we
3023 3022 * performed.
3024 3023 */
3025 3024 i40e_intr_io_disable_all(i40e);
3026 3025 i40e_intr_io_clear_cause(i40e);
3027 3026
3028 3027 if (i40e_shutdown_rings(i40e) == B_FALSE) {
3029 3028 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
3030 3029 }
3031 3030
3032 3031 delay(50 * drv_usectohz(1000));
3033 3032
3034 3033 /*
3035 3034 * We don't delete the default VSI because it replaces the VEB
3036 3035 * after VEB deletion (see the "Delete Element" section).
3037 3036 * Furthermore, since the default VSI is provided by the
3038 3037 * firmware, we never attempt to delete it.
3039 3038 */
3040 3039 for (i = 1; i < i40e->i40e_num_rx_groups; i++) {
3041 3040 i40e_delete_vsi(i40e, i);
3042 3041 }
3043 3042
3044 3043 if (i40e->i40e_veb_seid != 0) {
3045 3044 int rc = i40e_aq_delete_element(hw, i40e->i40e_veb_seid, NULL);
3046 3045
3047 3046 if (rc != I40E_SUCCESS) {
3048 3047 i40e_error(i40e, "Failed to delete VEB %d: %d", rc,
3049 3048 hw->aq.asq_last_status);
3050 3049 }
3051 3050
3052 3051 i40e->i40e_veb_seid = 0;
3053 3052 }
3054 3053
3055 3054 i40e_intr_chip_fini(i40e);
3056 3055
3057 3056 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
3058 3057 mutex_enter(&i40e->i40e_trqpairs[i].itrq_rx_lock);
3059 3058 mutex_enter(&i40e->i40e_trqpairs[i].itrq_tx_lock);
3060 3059 }
3061 3060
3062 3061 /*
3063 3062 * We should consider refactoring this to be part of the ring start /
3064 3063 * stop routines at some point.
3065 3064 */
3066 3065 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
3067 3066 i40e_stats_trqpair_fini(&i40e->i40e_trqpairs[i]);
3068 3067 }
3069 3068
3070 3069 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) !=
3071 3070 DDI_FM_OK) {
3072 3071 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
3073 3072 }
3074 3073
3075 3074 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
3076 3075 i40e_tx_cleanup_ring(&i40e->i40e_trqpairs[i]);
3077 3076 }
3078 3077
3079 3078 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
3080 3079 mutex_exit(&i40e->i40e_trqpairs[i].itrq_rx_lock);
3081 3080 mutex_exit(&i40e->i40e_trqpairs[i].itrq_tx_lock);
3082 3081 }
3083 3082
3084 3083 for (i = 0; i < i40e->i40e_num_rx_groups; i++) {
3085 3084 i40e_stat_vsi_fini(i40e, i);
3086 3085 }
3087 3086
3088 3087 i40e->i40e_link_speed = 0;
3089 3088 i40e->i40e_link_duplex = 0;
3090 3089 i40e_link_state_set(i40e, LINK_STATE_UNKNOWN);
3091 3090
3092 3091 if (free_allocations) {
3093 3092 i40e_free_ring_mem(i40e, B_FALSE);
3094 3093 }
3095 3094 }
3096 3095
3097 3096 boolean_t
3098 3097 i40e_start(i40e_t *i40e, boolean_t alloc)
3099 3098 {
3100 3099 i40e_hw_t *hw = &i40e->i40e_hw_space;
3101 3100 boolean_t rc = B_TRUE;
3102 3101 int i, err;
3103 3102
3104 3103 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
3105 3104
3106 3105 if (alloc) {
3107 3106 if (i40e_alloc_ring_mem(i40e) == B_FALSE) {
3108 3107 i40e_error(i40e,
3109 3108 "Failed to allocate ring memory");
3110 3109 return (B_FALSE);
3111 3110 }
3112 3111 }
3113 3112
3114 3113 /*
3115 3114 * This should get refactored to be part of ring start and stop at
3116 3115 * some point, along with most of the logic here.
3117 3116 */
3118 3117 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
3119 3118 if (i40e_stats_trqpair_init(&i40e->i40e_trqpairs[i]) ==
3120 3119 B_FALSE) {
3121 3120 int j;
3122 3121
3123 3122 for (j = 0; j < i; j++) {
3124 3123 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[j];
3125 3124 i40e_stats_trqpair_fini(itrq);
3126 3125 }
3127 3126 return (B_FALSE);
3128 3127 }
3129 3128 }
3130 3129
3131 3130 if (!i40e_chip_start(i40e)) {
3132 3131 i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE);
3133 3132 rc = B_FALSE;
3134 3133 goto done;
3135 3134 }
3136 3135
3137 3136 if (i40e_setup_rx_rings(i40e) == B_FALSE) {
3138 3137 rc = B_FALSE;
3139 3138 goto done;
3140 3139 }
3141 3140
3142 3141 if (i40e_setup_tx_rings(i40e) == B_FALSE) {
3143 3142 rc = B_FALSE;
3144 3143 goto done;
3145 3144 }
3146 3145
3147 3146 /*
3148 3147 * Enable broadcast traffic; however, do not enable multicast traffic.
3149 3148 * That's handle exclusively through MAC's mc_multicst routines.
3150 3149 */
3151 3150 err = i40e_aq_set_vsi_broadcast(hw, I40E_DEF_VSI_SEID(i40e), B_TRUE,
3152 3151 NULL);
3153 3152 if (err != I40E_SUCCESS) {
3154 3153 i40e_error(i40e, "failed to set default VSI: %d", err);
3155 3154 rc = B_FALSE;
3156 3155 goto done;
3157 3156 }
3158 3157
3159 3158 err = i40e_aq_set_mac_config(hw, i40e->i40e_frame_max, B_TRUE, 0, NULL);
3160 3159 if (err != I40E_SUCCESS) {
3161 3160 i40e_error(i40e, "failed to set MAC config: %d", err);
3162 3161 rc = B_FALSE;
3163 3162 goto done;
3164 3163 }
3165 3164
3166 3165 /*
3167 3166 * Finally, make sure that we're happy from an FM perspective.
3168 3167 */
3169 3168 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_reg_handle) !=
3170 3169 DDI_FM_OK) {
3171 3170 rc = B_FALSE;
3172 3171 goto done;
3173 3172 }
3174 3173
3175 3174 /* Clear state bits prior to final interrupt enabling. */
3176 3175 atomic_and_32(&i40e->i40e_state,
3177 3176 ~(I40E_ERROR | I40E_STALL | I40E_OVERTEMP));
3178 3177
3179 3178 i40e_intr_io_enable_all(i40e);
3180 3179
3181 3180 done:
3182 3181 if (rc == B_FALSE) {
3183 3182 i40e_stop(i40e, B_FALSE);
3184 3183 if (alloc == B_TRUE) {
3185 3184 i40e_free_ring_mem(i40e, B_TRUE);
3186 3185 }
3187 3186 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
3188 3187 }
3189 3188
3190 3189 return (rc);
3191 3190 }
3192 3191
3193 3192 /*
3194 3193 * We may have loaned up descriptors to the stack. As such, if we still have
3195 3194 * them outstanding, then we will not continue with detach.
3196 3195 */
3197 3196 static boolean_t
3198 3197 i40e_drain_rx(i40e_t *i40e)
3199 3198 {
3200 3199 mutex_enter(&i40e->i40e_rx_pending_lock);
3201 3200 while (i40e->i40e_rx_pending > 0) {
3202 3201 if (cv_reltimedwait(&i40e->i40e_rx_pending_cv,
3203 3202 &i40e->i40e_rx_pending_lock,
3204 3203 drv_usectohz(I40E_DRAIN_RX_WAIT), TR_CLOCK_TICK) == -1) {
3205 3204 mutex_exit(&i40e->i40e_rx_pending_lock);
3206 3205 return (B_FALSE);
3207 3206 }
3208 3207 }
3209 3208 mutex_exit(&i40e->i40e_rx_pending_lock);
3210 3209
3211 3210 return (B_TRUE);
3212 3211 }
3213 3212
3214 3213 /*
3215 3214 * DDI UFM Callbacks
3216 3215 */
3217 3216 static int
3218 3217 i40e_ufm_fill_image(ddi_ufm_handle_t *ufmh, void *arg, uint_t imgno,
3219 3218 ddi_ufm_image_t *img)
3220 3219 {
3221 3220 if (imgno != 0)
3222 3221 return (EINVAL);
3223 3222
3224 3223 ddi_ufm_image_set_desc(img, "Firmware");
3225 3224 ddi_ufm_image_set_nslots(img, 1);
3226 3225
3227 3226 return (0);
3228 3227 }
3229 3228
3230 3229 static int
3231 3230 i40e_ufm_fill_slot(ddi_ufm_handle_t *ufmh, void *arg, uint_t imgno,
3232 3231 uint_t slotno, ddi_ufm_slot_t *slot)
3233 3232 {
3234 3233 i40e_t *i40e = (i40e_t *)arg;
3235 3234 char *fw_ver = NULL, *fw_bld = NULL, *api_ver = NULL;
3236 3235 nvlist_t *misc = NULL;
3237 3236 uint_t flags = DDI_PROP_DONTPASS;
3238 3237 int err;
3239 3238
3240 3239 if (imgno != 0 || slotno != 0 ||
3241 3240 ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags,
3242 3241 "firmware-version", &fw_ver) != DDI_PROP_SUCCESS ||
3243 3242 ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags,
3244 3243 "firmware-build", &fw_bld) != DDI_PROP_SUCCESS ||
3245 3244 ddi_prop_lookup_string(DDI_DEV_T_ANY, i40e->i40e_dip, flags,
3246 3245 "api-version", &api_ver) != DDI_PROP_SUCCESS) {
3247 3246 err = EINVAL;
3248 3247 goto err;
3249 3248 }
3250 3249
3251 3250 ddi_ufm_slot_set_attrs(slot, DDI_UFM_ATTR_ACTIVE);
3252 3251 ddi_ufm_slot_set_version(slot, fw_ver);
3253 3252
3254 3253 (void) nvlist_alloc(&misc, NV_UNIQUE_NAME, KM_SLEEP);
3255 3254 if ((err = nvlist_add_string(misc, "firmware-build", fw_bld)) != 0 ||
3256 3255 (err = nvlist_add_string(misc, "api-version", api_ver)) != 0) {
3257 3256 goto err;
3258 3257 }
3259 3258 ddi_ufm_slot_set_misc(slot, misc);
3260 3259
3261 3260 ddi_prop_free(fw_ver);
3262 3261 ddi_prop_free(fw_bld);
3263 3262 ddi_prop_free(api_ver);
3264 3263
3265 3264 return (0);
3266 3265 err:
3267 3266 nvlist_free(misc);
3268 3267 if (fw_ver != NULL)
3269 3268 ddi_prop_free(fw_ver);
3270 3269 if (fw_bld != NULL)
3271 3270 ddi_prop_free(fw_bld);
3272 3271 if (api_ver != NULL)
3273 3272 ddi_prop_free(api_ver);
3274 3273
3275 3274 return (err);
3276 3275 }
3277 3276
3278 3277 static int
3279 3278 i40e_ufm_getcaps(ddi_ufm_handle_t *ufmh, void *arg, ddi_ufm_cap_t *caps)
3280 3279 {
3281 3280 *caps = DDI_UFM_CAP_REPORT;
3282 3281
3283 3282 return (0);
3284 3283 }
3285 3284
3286 3285 static ddi_ufm_ops_t i40e_ufm_ops = {
3287 3286 NULL,
3288 3287 i40e_ufm_fill_image,
3289 3288 i40e_ufm_fill_slot,
3290 3289 i40e_ufm_getcaps
3291 3290 };
3292 3291
3293 3292 static int
3294 3293 i40e_attach(dev_info_t *devinfo, ddi_attach_cmd_t cmd)
3295 3294 {
3296 3295 i40e_t *i40e;
3297 3296 struct i40e_osdep *osdep;
3298 3297 i40e_hw_t *hw;
3299 3298 int instance;
3300 3299
3301 3300 if (cmd != DDI_ATTACH)
3302 3301 return (DDI_FAILURE);
3303 3302
3304 3303 instance = ddi_get_instance(devinfo);
3305 3304 i40e = kmem_zalloc(sizeof (i40e_t), KM_SLEEP);
3306 3305
3307 3306 i40e->i40e_aqbuf = kmem_zalloc(I40E_ADMINQ_BUFSZ, KM_SLEEP);
3308 3307 i40e->i40e_instance = instance;
3309 3308 i40e->i40e_dip = devinfo;
3310 3309
3311 3310 hw = &i40e->i40e_hw_space;
3312 3311 osdep = &i40e->i40e_osdep_space;
3313 3312 hw->back = osdep;
3314 3313 osdep->ios_i40e = i40e;
3315 3314
3316 3315 ddi_set_driver_private(devinfo, i40e);
3317 3316
3318 3317 i40e_fm_init(i40e);
3319 3318 i40e->i40e_attach_progress |= I40E_ATTACH_FM_INIT;
3320 3319
3321 3320 if (pci_config_setup(devinfo, &osdep->ios_cfg_handle) != DDI_SUCCESS) {
3322 3321 i40e_error(i40e, "Failed to map PCI configurations.");
3323 3322 goto attach_fail;
3324 3323 }
3325 3324 i40e->i40e_attach_progress |= I40E_ATTACH_PCI_CONFIG;
3326 3325
3327 3326 i40e_identify_hardware(i40e);
3328 3327
3329 3328 if (!i40e_regs_map(i40e)) {
3330 3329 i40e_error(i40e, "Failed to map device registers.");
3331 3330 goto attach_fail;
3332 3331 }
3333 3332 i40e->i40e_attach_progress |= I40E_ATTACH_REGS_MAP;
3334 3333
3335 3334 i40e_init_properties(i40e);
3336 3335 i40e->i40e_attach_progress |= I40E_ATTACH_PROPS;
3337 3336
3338 3337 if (!i40e_common_code_init(i40e, hw))
3339 3338 goto attach_fail;
3340 3339 i40e->i40e_attach_progress |= I40E_ATTACH_COMMON_CODE;
3341 3340
3342 3341 /*
3343 3342 * When we participate in IRM, we should make sure that we register
3344 3343 * ourselves with it before callbacks.
3345 3344 */
3346 3345 if (!i40e_alloc_intrs(i40e, devinfo)) {
3347 3346 i40e_error(i40e, "Failed to allocate interrupts.");
3348 3347 goto attach_fail;
3349 3348 }
3350 3349 i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_INTR;
3351 3350
3352 3351 if (!i40e_alloc_trqpairs(i40e)) {
3353 3352 i40e_error(i40e,
3354 3353 "Failed to allocate receive & transmit rings.");
3355 3354 goto attach_fail;
3356 3355 }
3357 3356 i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_RINGSLOCKS;
3358 3357
3359 3358 if (!i40e_map_intrs_to_vectors(i40e)) {
3360 3359 i40e_error(i40e, "Failed to map interrupts to vectors.");
3361 3360 goto attach_fail;
3362 3361 }
3363 3362
3364 3363 if (!i40e_add_intr_handlers(i40e)) {
3365 3364 i40e_error(i40e, "Failed to add the interrupt handlers.");
3366 3365 goto attach_fail;
3367 3366 }
3368 3367 i40e->i40e_attach_progress |= I40E_ATTACH_ADD_INTR;
3369 3368
3370 3369 if (!i40e_final_init(i40e)) {
3371 3370 i40e_error(i40e, "Final initialization failed.");
3372 3371 goto attach_fail;
3373 3372 }
3374 3373 i40e->i40e_attach_progress |= I40E_ATTACH_INIT;
3375 3374
3376 3375 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) !=
3377 3376 DDI_FM_OK) {
3378 3377 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
3379 3378 goto attach_fail;
3380 3379 }
3381 3380
3382 3381 if (!i40e_stats_init(i40e)) {
3383 3382 i40e_error(i40e, "Stats initialization failed.");
3384 3383 goto attach_fail;
3385 3384 }
3386 3385 i40e->i40e_attach_progress |= I40E_ATTACH_STATS;
3387 3386
3388 3387 if (!i40e_register_mac(i40e)) {
3389 3388 i40e_error(i40e, "Failed to register to MAC/GLDv3");
3390 3389 goto attach_fail;
3391 3390 }
3392 3391 i40e->i40e_attach_progress |= I40E_ATTACH_MAC;
3393 3392
3394 3393 i40e->i40e_periodic_id = ddi_periodic_add(i40e_timer, i40e,
3395 3394 I40E_CYCLIC_PERIOD, DDI_IPL_0);
3396 3395 if (i40e->i40e_periodic_id == 0) {
3397 3396 i40e_error(i40e, "Failed to add the link-check timer");
3398 3397 goto attach_fail;
3399 3398 }
3400 3399 i40e->i40e_attach_progress |= I40E_ATTACH_LINK_TIMER;
3401 3400
3402 3401 if (!i40e_enable_interrupts(i40e)) {
3403 3402 i40e_error(i40e, "Failed to enable DDI interrupts");
3404 3403 goto attach_fail;
3405 3404 }
3406 3405 i40e->i40e_attach_progress |= I40E_ATTACH_ENABLE_INTR;
3407 3406
3408 3407 if (ddi_ufm_init(i40e->i40e_dip, DDI_UFM_CURRENT_VERSION, &i40e_ufm_ops,
3409 3408 &i40e->i40e_ufmh, i40e) != 0) {
3410 3409 i40e_error(i40e, "failed to initialize UFM subsystem");
3411 3410 goto attach_fail;
3412 3411 }
3413 3412 ddi_ufm_update(i40e->i40e_ufmh);
3414 3413 i40e->i40e_attach_progress |= I40E_ATTACH_UFM_INIT;
3415 3414
3416 3415 atomic_or_32(&i40e->i40e_state, I40E_INITIALIZED);
3417 3416
3418 3417 mutex_enter(&i40e_glock);
3419 3418 list_insert_tail(&i40e_glist, i40e);
3420 3419 mutex_exit(&i40e_glock);
3421 3420
3422 3421 return (DDI_SUCCESS);
3423 3422
3424 3423 attach_fail:
3425 3424 i40e_unconfigure(devinfo, i40e);
3426 3425 return (DDI_FAILURE);
3427 3426 }
3428 3427
3429 3428 static int
3430 3429 i40e_detach(dev_info_t *devinfo, ddi_detach_cmd_t cmd)
3431 3430 {
3432 3431 i40e_t *i40e;
3433 3432
3434 3433 if (cmd != DDI_DETACH)
3435 3434 return (DDI_FAILURE);
3436 3435
3437 3436 i40e = (i40e_t *)ddi_get_driver_private(devinfo);
3438 3437 if (i40e == NULL) {
3439 3438 i40e_log(NULL, "i40e_detach() called with no i40e pointer!");
3440 3439 return (DDI_FAILURE);
3441 3440 }
3442 3441
3443 3442 if (i40e_drain_rx(i40e) == B_FALSE) {
3444 3443 i40e_log(i40e, "timed out draining DMA resources, %d buffers "
3445 3444 "remain", i40e->i40e_rx_pending);
3446 3445 return (DDI_FAILURE);
3447 3446 }
3448 3447
3449 3448 mutex_enter(&i40e_glock);
3450 3449 list_remove(&i40e_glist, i40e);
3451 3450 mutex_exit(&i40e_glock);
3452 3451
3453 3452 i40e_unconfigure(devinfo, i40e);
3454 3453
3455 3454 return (DDI_SUCCESS);
3456 3455 }
3457 3456
3458 3457 static struct cb_ops i40e_cb_ops = {
3459 3458 nulldev, /* cb_open */
3460 3459 nulldev, /* cb_close */
3461 3460 nodev, /* cb_strategy */
3462 3461 nodev, /* cb_print */
3463 3462 nodev, /* cb_dump */
3464 3463 nodev, /* cb_read */
3465 3464 nodev, /* cb_write */
3466 3465 nodev, /* cb_ioctl */
3467 3466 nodev, /* cb_devmap */
3468 3467 nodev, /* cb_mmap */
3469 3468 nodev, /* cb_segmap */
3470 3469 nochpoll, /* cb_chpoll */
3471 3470 ddi_prop_op, /* cb_prop_op */
3472 3471 NULL, /* cb_stream */
3473 3472 D_MP | D_HOTPLUG, /* cb_flag */
3474 3473 CB_REV, /* cb_rev */
3475 3474 nodev, /* cb_aread */
3476 3475 nodev /* cb_awrite */
3477 3476 };
3478 3477
3479 3478 static struct dev_ops i40e_dev_ops = {
3480 3479 DEVO_REV, /* devo_rev */
3481 3480 0, /* devo_refcnt */
3482 3481 NULL, /* devo_getinfo */
3483 3482 nulldev, /* devo_identify */
3484 3483 nulldev, /* devo_probe */
3485 3484 i40e_attach, /* devo_attach */
3486 3485 i40e_detach, /* devo_detach */
3487 3486 nodev, /* devo_reset */
3488 3487 &i40e_cb_ops, /* devo_cb_ops */
3489 3488 NULL, /* devo_bus_ops */
3490 3489 ddi_power, /* devo_power */
3491 3490 ddi_quiesce_not_supported /* devo_quiesce */
3492 3491 };
3493 3492
3494 3493 static struct modldrv i40e_modldrv = {
3495 3494 &mod_driverops,
3496 3495 i40e_ident,
3497 3496 &i40e_dev_ops
3498 3497 };
3499 3498
3500 3499 static struct modlinkage i40e_modlinkage = {
3501 3500 MODREV_1,
3502 3501 &i40e_modldrv,
3503 3502 NULL
3504 3503 };
3505 3504
3506 3505 /*
3507 3506 * Module Initialization Functions.
3508 3507 */
3509 3508 int
3510 3509 _init(void)
3511 3510 {
3512 3511 int status;
3513 3512
3514 3513 list_create(&i40e_glist, sizeof (i40e_t), offsetof(i40e_t, i40e_glink));
3515 3514 list_create(&i40e_dlist, sizeof (i40e_device_t),
3516 3515 offsetof(i40e_device_t, id_link));
3517 3516 mutex_init(&i40e_glock, NULL, MUTEX_DRIVER, NULL);
3518 3517 mac_init_ops(&i40e_dev_ops, I40E_MODULE_NAME);
3519 3518
3520 3519 status = mod_install(&i40e_modlinkage);
3521 3520 if (status != DDI_SUCCESS) {
3522 3521 mac_fini_ops(&i40e_dev_ops);
3523 3522 mutex_destroy(&i40e_glock);
3524 3523 list_destroy(&i40e_dlist);
3525 3524 list_destroy(&i40e_glist);
3526 3525 }
3527 3526
3528 3527 return (status);
3529 3528 }
3530 3529
3531 3530 int
3532 3531 _info(struct modinfo *modinfop)
3533 3532 {
3534 3533 return (mod_info(&i40e_modlinkage, modinfop));
3535 3534 }
3536 3535
3537 3536 int
3538 3537 _fini(void)
3539 3538 {
3540 3539 int status;
3541 3540
3542 3541 status = mod_remove(&i40e_modlinkage);
3543 3542 if (status == DDI_SUCCESS) {
3544 3543 mac_fini_ops(&i40e_dev_ops);
3545 3544 mutex_destroy(&i40e_glock);
3546 3545 list_destroy(&i40e_dlist);
3547 3546 list_destroy(&i40e_glist);
3548 3547 }
3549 3548
3550 3549 return (status);
3551 3550 }
|
↓ open down ↓ |
1854 lines elided |
↑ open up ↑ |
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX