Many 10GbE DACs can be hacked to run 25GbE, which is something I had to figure out after realizing that it’s not just plug and play as it was with twisted pair cables coming from 1Gb to 2.5Gb…
I'm waiting for 3 DACs and a few other bits to arrive today to move closer to 10G networking at home. Moving house soon and the new place will have 2.5Gbps FTTP (both up and down) so I wanted to be prepared for that. Given my existing broadband is only 500/75Mbps FTTP I was fine with a 1GbE internal network and Wifi-6 meshing. I could have planned to move to 2.5GbE but it may have been a bottleneck at some point, so may as well push straight on to 10G.
I have a USW-Aggregation with 8 SFP+ ports arriving today too. Just have to install Intel X520-DA2 cards in two of my servers (Proxmox host and a general Linux server), and the NAS also has a 10G SFP+ port, and then connect it all up.
Most of it second hand from eBay for half the usual retail price.
I did similar with the Mikrotik CRS305-1G-4S+IN and some surplus eBay gear. The nice thing is the NAS and my MacBook dock both have 10G and are connected - and it’s noticeable.
The card is obviously 16-lane, but it also has two ports; 40Gb total. In a server that’s fine, but if you want 10G in a desktop you’ll have a problem.
I’m probably not telling you anything new. NICs using newer PCI generations are rare as hen’s teeth. It should be possible to do this with four lanes, but isn’t…
Unless you find a 25G dual-port card, in which case the single lane my secondary slots hand out does at least suffice for 10G one way.
I’m a huge fan of fiber optic cables instead of DAC. For one thing, the distance you can run is far greater. For another, they don’t get as hot as metal cables, particularly at high speeds. OM3 and OM4 is actually quite affordable and SFP+ is nice.
Yes on an absolute sense, DAC can be cheaper, but for a homelab or something, it’s not that much different. Also, it’s much cooler to run fiber optic to things (imho).
Surprised it doesn't mention heat - one of the big advantages.
There is also a new generation of "invisible" fiber that is interesting for home use. Basically looks like fishing line so you can run it somewhere that you just don't see unless you're really looking. Fragile tho
While working for a broadcasting/networking integration gig, we designed, assembled and field-terminated thousands of copper and fibre runs from 1/2 ft to football field lengths.
The amount of time required to terminate a copper cable in the field is seconds, and felt a bit like art. Something about the way it reliably reacted was magical and felt "strong."
Terminating or splicing a fibre cable felt like wrestling a snake covered in melted crayons, and the failure rate was significantly higher across the board. And it wasn't just workmanship, but quality of product, terminating environment, available equipment, misuse by future operators etc.
That said, at a certain point, we as a firm learned that most purchasers would rather the low latency/small footprint of optical/fibre versus copper, maintenece/failure be damned. Though, maybe part of our willingness to push fibre came from knowing that most purchasers would in 1-2 years call us back in to replace the rack terminations with copper :)
There are 2 very different main types of "copper" in this context. Each can break down into more specific subcategories but they have a relatively common general behavior with their primary type.
First there is the BASE-T RJ45 stuff, which it sounds like you might have been working with. At 10G or higher speeds this get relatively power hungry and is not really an advantage over fiber unless you are also delivering PoE or are trying to reuse existing cabling.
This type (DAC) is a special type of pre-made cable assembly which eschews much of the advanced signalling/conversion logic. The upside is the power usage is low (often even lower than fiber) and the cost is dirt cheap. The downside is the lengths are much more limited and it's intended to be preterminated SFP-to-SFP (or the like) cable assemblies instead of modular patching/custom built.
There is indeed a latency difference, usually DAC < fiber < BASE-T, but they are all within a few microseconds (not milliseconds) of each other so you really have to be pressed to care about it (to the point you're looking at specialized low latency switches and paying extra to lay things out in a way which minimizes the number of L2 hops rather than the cost).
> That said, at a certain point, we as a firm learned that most purchasers would rather the low latency/small footprint of optical/fibre versus copper, maintenece/failure be damned.
Copper bundles get real thick, real fast: I ran an OneFS cluster for many years, and we had >50 nodes, and all the cables (each node dual-connected) ran to two central switches for backend replications. Rat's nest.
I was very happy when Isilon started officially supporting active optical cable (AOC) on the backend. Really helped with airflow and keeping things tidy.
> This is a bit dependent on the type of DAC, and the vendor. Generally speaking, we are moving into an era as we move above 100GbE speeds where DACs will be limited to 5m runs and shorter runs.
This is generally good enough for intra-rack runs, especially if you put your top-of-rack (ToR) switch in the middle: you basically only have to go about 1m/3' in either direction (up/down).
Dacs are awesome until you encounter a full rack with nothing but 2 and 4 meter dacs. They appeal very much to the lazy engineer, and a lazy engineer can turn them into a rats nest.
Many 10GbE DACs can be hacked to run 25GbE, which is something I had to figure out after realizing that it’s not just plug and play as it was with twisted pair cables coming from 1Gb to 2.5Gb…
https://kohlschuetter.github.io/blog/posts/2026/03/22/unlock...
I'm waiting for 3 DACs and a few other bits to arrive today to move closer to 10G networking at home. Moving house soon and the new place will have 2.5Gbps FTTP (both up and down) so I wanted to be prepared for that. Given my existing broadband is only 500/75Mbps FTTP I was fine with a 1GbE internal network and Wifi-6 meshing. I could have planned to move to 2.5GbE but it may have been a bottleneck at some point, so may as well push straight on to 10G.
I have a USW-Aggregation with 8 SFP+ ports arriving today too. Just have to install Intel X520-DA2 cards in two of my servers (Proxmox host and a general Linux server), and the NAS also has a 10G SFP+ port, and then connect it all up.
Most of it second hand from eBay for half the usual retail price.
I did similar with the Mikrotik CRS305-1G-4S+IN and some surplus eBay gear. The nice thing is the NAS and my MacBook dock both have 10G and are connected - and it’s noticeable.
Nice work, that agg switch is excelllwnt.
I went with some cheap eBay cards and slotted them into a synology and PC.
They work great and have for years.
https://www.ebay.com/itm/384094168784?_skw=connectx+mellanox...
How many PCIe lanes are you allocating?
The card is obviously 16-lane, but it also has two ports; 40Gb total. In a server that’s fine, but if you want 10G in a desktop you’ll have a problem.
I’m probably not telling you anything new. NICs using newer PCI generations are rare as hen’s teeth. It should be possible to do this with four lanes, but isn’t…
Unless you find a 25G dual-port card, in which case the single lane my secondary slots hand out does at least suffice for 10G one way.
I’m a huge fan of fiber optic cables instead of DAC. For one thing, the distance you can run is far greater. For another, they don’t get as hot as metal cables, particularly at high speeds. OM3 and OM4 is actually quite affordable and SFP+ is nice.
Yes on an absolute sense, DAC can be cheaper, but for a homelab or something, it’s not that much different. Also, it’s much cooler to run fiber optic to things (imho).
Surprised it doesn't mention heat - one of the big advantages.
There is also a new generation of "invisible" fiber that is interesting for home use. Basically looks like fishing line so you can run it somewhere that you just don't see unless you're really looking. Fragile tho
While working for a broadcasting/networking integration gig, we designed, assembled and field-terminated thousands of copper and fibre runs from 1/2 ft to football field lengths.
The amount of time required to terminate a copper cable in the field is seconds, and felt a bit like art. Something about the way it reliably reacted was magical and felt "strong."
Terminating or splicing a fibre cable felt like wrestling a snake covered in melted crayons, and the failure rate was significantly higher across the board. And it wasn't just workmanship, but quality of product, terminating environment, available equipment, misuse by future operators etc.
That said, at a certain point, we as a firm learned that most purchasers would rather the low latency/small footprint of optical/fibre versus copper, maintenece/failure be damned. Though, maybe part of our willingness to push fibre came from knowing that most purchasers would in 1-2 years call us back in to replace the rack terminations with copper :)
There are 2 very different main types of "copper" in this context. Each can break down into more specific subcategories but they have a relatively common general behavior with their primary type.
First there is the BASE-T RJ45 stuff, which it sounds like you might have been working with. At 10G or higher speeds this get relatively power hungry and is not really an advantage over fiber unless you are also delivering PoE or are trying to reuse existing cabling.
This type (DAC) is a special type of pre-made cable assembly which eschews much of the advanced signalling/conversion logic. The upside is the power usage is low (often even lower than fiber) and the cost is dirt cheap. The downside is the lengths are much more limited and it's intended to be preterminated SFP-to-SFP (or the like) cable assemblies instead of modular patching/custom built.
There is indeed a latency difference, usually DAC < fiber < BASE-T, but they are all within a few microseconds (not milliseconds) of each other so you really have to be pressed to care about it (to the point you're looking at specialized low latency switches and paying extra to lay things out in a way which minimizes the number of L2 hops rather than the cost).
> That said, at a certain point, we as a firm learned that most purchasers would rather the low latency/small footprint of optical/fibre versus copper, maintenece/failure be damned.
Copper bundles get real thick, real fast: I ran an OneFS cluster for many years, and we had >50 nodes, and all the cables (each node dual-connected) ran to two central switches for backend replications. Rat's nest.
I was very happy when Isilon started officially supporting active optical cable (AOC) on the backend. Really helped with airflow and keeping things tidy.
> This is a bit dependent on the type of DAC, and the vendor. Generally speaking, we are moving into an era as we move above 100GbE speeds where DACs will be limited to 5m runs and shorter runs.
This is generally good enough for intra-rack runs, especially if you put your top-of-rack (ToR) switch in the middle: you basically only have to go about 1m/3' in either direction (up/down).
Dacs are awesome until you encounter a full rack with nothing but 2 and 4 meter dacs. They appeal very much to the lazy engineer, and a lazy engineer can turn them into a rats nest.
To be fair, the dedicated lazy engineer can make a rats nest out of any interconnect technology.
Dacs are great … they let you avoid having to convert to optical or having to splice your own cable.
Downside the cables are kind of heavy and sometimes folks dont like that.