ePIC pfRICH electronics mailing list

[Miklos Czeller <miklos.czeller AT cern.ch>]

Hi Alex,

Currently I am in Chicago in RSNA, so sorry for the slow reaction.

The final ethernet solution is not ready. I don't know the final power estimation now. 

But I created and translated 3 reference designs at night which are very similar than I want to write. 

In all cases the power estimation was below 0.7W.

My previous estimation was 2W + 10%. Now it seems this is a little bit bigger. This is depends from clocking...

We should be somewhere around 2.7W. But it is only a calculation based on Vivado.

Cheers,

M!klos

 

---

From: Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l-bounces AT lists.bnl.gov> on behalf of Kiselev, Alexander via Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Sent: Monday, November 27, 2023 2:34 PM
To: Gabor Nagy <hunagabo AT gmail.com>
Cc: eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

  Hi Gabor,

  Thank you for sharing, this certainly looks promising. As Miklos pointed out earlier, we are not going to use DHCP. Miklos: do I get it right, that with static IP addresses, the power consumption estimate as described in section 6 of the interface document, remains valid?

  Concerning the layout of the FPGA board in general, we should probably take a step back and re-discuss the boundary conditions. Remember, we wanted these FPGA boards to be short on the right in order not to be in a conflict with the ASIC board vertical LV connectors (which are now optional, and horizontal anyway). And there was never a problem to have these board longer than the FMC ones on the left side. Having any vertical connectors of the FPGA board below the fan assembly will be very inconvenient, I'm afraid. In the present design this is true for USB & JTAG ones, but most likely for the ethernet connector as well. 

  I guess my question (primarily to Pierrick and Daniel) is: are there any reasons against making these boards 119mm long? In which case (either already in V0 or perhaps only in V1) could the JTAG and ethernet connectors be moved to the far left, and a vertical USB connector with its support circuitry be moved to the far right (and in both cases be safely outside of the fan assembly footprint)?

  Cheers,

    Alexander.

 

 

---

From: nagy g <hunagabo AT gmail.com>
Sent: Sunday, November 26, 2023 4:37 PM
To: Kiselev, Alexander <ayk AT bnl.gov>
Cc: Cacace, Daniel <dcacace AT bnl.gov>; eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>; Christophe de La Taille <taille AT in2p3.fr>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

Hello Alexander,

Miklos is working on the Ethernet code, so hopefully in next weeks we will get a more reliable estimate for the Ethernet code as well. Unfortunately my latest experinec with Ethernet code was 6 years ago with a Kintex7 FPGA and that is probably not relevant. (and that included several protocol imnplementation like DHCp..).

So, I will be more carefull with worst case estimates...

I was working on the FPGA board design and there are few thing I would share (just facts:)).

1. it looks like the safety plan need to be changed a bit, because with the two additional hole for the heatsink and all FPGA related power supplies/components I can not add the USB part directly to the PCB:

Unfortunately most of the connectors has mounting holes(s) which can not collide even we use HDI (High Density Interconnect) technology and put them on the opposite side.

But we discussed it with Miklos, and the conclusion was that (as he wants to keep this safety function), an extra FFC connector can be added, which allows external USB-FIFO board to be connected if that really necessary.

So, if you agree, that will be the new safety plan.

2. We discussed that earlier, the normal Ethernet connector with the internal transformers are barelly fit to the board (with all mounting holes of the ERM5 connectors and screws...), so, the RJ45 connector changed to MiniIO Type I connector what you mentioned earlier.

It can be placed with the transformers as well "much" easier.

TE has a standard cable for this connector in 1,2,3..10m length:

https://www.te.com/usa-en/product-2-2205133-2.html

https://www.te.com/usa-en/product-2-2205133-3.html

https://www.te.com/usa-en/product-2-2205133-4.html

And some of them available at distributors:

https://hu.mouser.com/ProductDetail/TE-Connectivity/2-2205133-3?qs=lyRJTFulC0a2MX%252BarADDig%3D%3D

https://www.digikey.hu/en/products/detail/te-connectivity-amp-connectors/2-2205133-6/6042748

https://hu.farnell.com/te-connectivity/2-2205133-5/patch-cord-mini-io-type-i-rj45/dp/2672836

...

3. As the space is highly fragmented with the mounting holes, I could not use the uppr left side of the PCB for too much, but fortunately the mounting holes of the standard 2mm pitch JTAG connector just fit into the ERM5 connector midle, I could add that connector:

Hopefully it helps in debugging and not just makes the assembly process more difficult...

So, even this version is far away from the final one (decoupling and power components need to be moved around..) I generated a STEP file from the current version. Hopefully the components are not changing too much from now just the position of them...

The step file is available here: https://drive.google.com/file/d/1_oNf-7kvGNoB3uz-9coUAlXWRz1acHvL/view?usp=drive_link

Best regards, gabor

Kiselev, Alexander <ayk AT bnl.gov> ezt írta (időpont: 2023. nov. 24., P, 15:40):

  Dear Gabor,

  the biggest discrepancy comes from adding ethernet functionality to the FPGA codes, as far as I can tell. It was presently estimated as a "safety margin of +10%" to the plot shown in section 6. Can we please clarify this? Or otherwise we will have to ask Daniel to do a FEA evaluation for two cases (there is a substantial difference between <30W and ~50W I guess). 

  Cheers,

    Alexander.

---

From: nagy g <hunagabo AT gmail.com>
Sent: Wednesday, November 22, 2023 1:14 PM
To: Kiselev, Alexander <ayk AT bnl.gov>
Cc: Cacace, Daniel <dcacace AT bnl.gov>; eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>; Christophe de La Taille <taille AT in2p3.fr>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

Hi Alexander,

Originally, I calculated with 1A+1A for each ASIC and the FMC power suplies have designed for that. Two weeks ago Pierrick confirmed that the total current is the 1A which is halfed the current what the FMC/FPGA card need to provide for the ASICs board. It could be my missunderstanding, but that happened.

I tried to provide the cealing in my power estimate, and provided a very pessimistic FPGA budget, but once we have a schematic, and a VHDL code close to the final version, we can provide a much more accurate numbers (regarding the FPGA board).

Best Regards, gabor

Kiselev, Alexander <ayk AT bnl.gov> ezt írta (időpont: 2023. nov. 22., Sze, 15:43):

  Hi Gabor, All,

  What you are writing about the FPGA boards is in a drastic disagreement with what was distributed in the latest version of our interface document, see attached, section 6. Friends, we should please concentrate on this topic, and take it seriously, eventually. There is no way we can operate this setup at any temperature higher than 50C, and even that would be dangerous for the sensors (the indium seal in their current implementation just melts at 72C, and who knows what happens if there is a substantial temperature gradient across the sensor package). That's why the temperature control should be bullet proof as well, otherwise we are at a risk of destroying $200k worth of equipment in one shot. Of course, 50C at the ASIC location does not mean 50C at where the sensor window seal is, but you see the point.

  So can we please straighten the numbers up, and agree on how much each ASIC will consume if no TOT functionality is used, and most part of channels 64..71 is turned off (Damien), and whether the ethernet functionality in a FPGA adds a little or a lot to the power budget (Miklos, Gabor). And if we are not sure where is it going in terms of the present cooling setup efficiency in general, should not we bite the bullet and reconsider it from scratch (Daniel), perhaps providing all the participating groups with some sort of a water cooling equipment by the time of  the V1 iteration.

  Regards,

    Alexander.

---

From: nagy g <hunagabo AT gmail.com>
Sent: Wednesday, November 22, 2023 8:40 AM
To: Kiselev, Alexander <ayk AT bnl.gov>
Cc: Cacace, Daniel <dcacace AT bnl.gov>; eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>; Christophe de La Taille <taille AT in2p3.fr>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

Hi Daniel,

It is highly depends on the ASICs power dissipation what I do not see now (maybe Damien can give a more precise numbers)... for the FMC card we got a 1A / ASIC estimation, which is 1.2W*8 ASICs = 9.6W plus the 10% power loss of the DCDC converters which is ~1W more. Additionally we have power "filters" (LDO regulators) for the analog supply which is 0.15*Analog current of the ASICs which is in the worst-worst case 8A. So we are we do not expect more than 12W for the V0 system ( per half AISCs board).

But hopefully this can be lowered with switching off not used parts of the ASICs.

For the FPGA version (V1) we have Ethernet + FPGA + 3 more Power supplies. The FPGA (with the current code) estimated to dissipate ~2W, but I don't think it is close to the final number, so I would expect ~8-10W for final code with ethernet functions.

The Ethernet PHY is dissipating max 663mW and the other (pull-ups/downs, filters...) is n*100mW which is probably below 1W.

Plus the 10% of DCDC supplies, so it is another ~12W

So for the full ASICs board (booth half) with the two FPGA card it is ~48W with the worst case calculation.

The components are commercial grade ones which means max +85C working temperature. (I am not sure what would be the normal temp range for the ASICs, but I guess we do not want to go over 30-35C)

It's my two cents...:)

Best regards, gabor

Kiselev, Alexander via Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l AT lists.bnl.gov> ezt írta (időpont: 2023. nov. 21., K, 2:51):

  Hi Daniel,

  this is a question to Gabor I think. We estimated the total power consumption to be below 30W (perhaps 20-25W if part of the ASIC functionality is turned off), there must be at least one of these regulators per FMC board, and power efficiency was supposed to be >90%. Sounds like ~1W or so of power per spot to me?

  Cheers,

    Alexander.

---

From: Cacace, Daniel <dcacace AT bnl.gov>
Sent: Monday, November 20, 2023 3:10 PM
To: eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>; Christophe de La Taille <taille AT in2p3.fr>; Kiselev, Alexander <ayk AT bnl.gov>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

Hi Alexander,

How much heat are we talking about and what temp do you want them? If it's not very much, we can use something like this: https://www.mcmaster.com/8822T912/

Cheers,

Dan Cacace
sPHENIX & ePIC
Physics Department
Office: 631.344.2197
dcacace AT bnl.gov

---

From: Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l-bounces AT lists.bnl.gov> on behalf of Kiselev, Alexander via Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Sent: Wednesday, November 15, 2023 7:55 AM
To: eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>; Christophe de La Taille <taille AT in2p3.fr>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

  Dear Christophe, All,

  Concerning the unification of power connectors, after our Friday meeting I also realized that this was not a good idea, even if power levels were the same on both types of boards (because one can still accidentally change a correct connection when two FPGA boards are powered at once into an incorrect one when ASIC and FPGA boards are powered at the same time). Somehow I had a false "one cable only" picture in mind when proposing this unification. If however you were going to feed 1.2V to the ASIC board, we absolutely should not use the same connector, up to the point that the pitch should better be chosen differently (or otherwise one should not be technically able to plug a cord in a wrong place).

  -> Pierrick, will you choose a different one on your board? Or we ask Gabor to make a change?

  Concerning noise, will the heat sink itself (in FPGA configuration only) be usable as a shield? See the picture attached. It will be grounded by a braid to the ASIC board itself, and all the ASICs will be sitting under the aluminum plate. One can think of building one more sink like this, with a cutaway for a Twinax cable (then usable on your test stand with the FMC board as well). I'm rather reluctant to change Gabor's V0 board design now, to move the regulators. This can later be fixed in V1, or (if absolutely needed) in V0 FPGA board, correct?

  Concerning overheating, Daniel: can we use thick gap pads you showed me once, so that there is a thermal contact between the regulators and one of the neighboring heat sinks?

  Cheers,

    Alexander.

---

From: Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l-bounces AT lists.bnl.gov> on behalf of Christophe de La Taille via Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Sent: Wednesday, November 15, 2023 4:34 AM
To: eic-projdet-pfrich-electronics-l AT lists.bnl.gov <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Subject: Re: [Eic-projdet-pfrich-electronics-l] HRPPD HGCROC backplane interface document r12

 

dear all,

looking at the boards and schematics with Pierrick and Damien, we are a bit worried to use the same connector for the direct ASIC power supply and for the FPGA as a risk of bringing 5V to the ASICs if plugged in the wrong place, this would destroy all the chips.  We could use 2x2 and a 1x4 in the different locations to minimize this risk or avoid mounting them on the ASIC board if the powering via FMC/FPGA board is OK.

The DCDC convertor can also be a source of noise as the chips inputs are very sensitive. On CMS hexaboards, the convertors have to be shielded and we saw on the noise the place where there was a hole in the shield... Facing the chips rather than the outside may also increase the risk of noise coupling.  Providing possibility of mounting a shield, thoroughly connected to ground would be useful.

The heat dissipation of both DCDC and linear regulators may also be an issue, are you confident that they will not overheat ?

Probably we can proceed as it is now in order not to delay the schedule and keep it for next version.

Best regards :  Christophe

On 14/11/2023 15:43, Kiselev, Alexander via Eic-projdet-pfrich-electronics-l wrote:

  Hello colleagues,

  attached is r14, with the recent corrections by Gabor implemented, and hyperlinks working.

  Please provide your feedback by tomorrow Wednesday. I'd also appreciate if people address the highlighted items by the end of this week, except for sections 5 & 6.

  Regards,

    Alexander.

---

From: Kiselev, Alexander
Sent: Monday, November 13, 2023 2:59 AM
To: Camarda, Timothy via Eic-projdet-pfrich-electronics-l <eic-projdet-pfrich-electronics-l AT lists.bnl.gov>
Subject: HRPPD HGCROC backplane interface document r12

 

  Hello colleagues,

  r12 with the modifications following our Friday meeting is attached. I believe I incorporated all comments, and also uploaded all new materials.

  Missing items are color coded: magenta (AK), red (Pierrick), cyan (Gabor), yellow (Miklos), green (Daniel). Some of them require more work to converge.

  Daniel: once Pierrick provides his CAD model, please perform a final consistency check, at which point one can start finalizing the cooling system configuration.

  All those with experience in schematics (but Pierrick in particular): please consider providing your feedback to the FMC board package revision r03 which Gabor sent out yesterday (it is also linked from the attached pdf file), by Wednesday.

  Regards,

    Alexander.

-- 
--
Christophe de LA TAILLE
OMEGA CNRS/IN2P3 Micro-Electronics Design Lab
Professor of micro-electronics at Ecole Polytechnique
Ecole Polytechnique  F91128 Palaiseau France
+33 16933 8998

--
Eic-projdet-pfrich-electronics-l mailing list
Eic-projdet-pfrich-electronics-l AT lists.bnl.gov
https://lists.bnl.gov/mailman/listinfo/eic-projdet-pfrich-electronics-l