sPHENIX MAPS tracker discussion

[Ming Liu <ming AT bnl.gov>]

FYI,
More recent discussions regarding FLX712 driver and FW ... 

Ming 

-- 
Ming Xiong Liu
P-25, MS H846                     TEL: 505-667-7125 
Physics Division                            631-344-7821(BNL)
LANL                                               630-840-5708(FNAL)
Los Alamos, NM 87545      FAX:  505-665-7020
 

On 11/2/20, 9:42 AM, "Philippe Laurens" <philippe.laurens AT cern.ch> wrote:

    Hello,

    We are adding a FLX712 to our test stand where we are trying to
    forward the TTC clock and trigger information on 2 optical links.
    We will not be trying to readout anything (at this time.)

    We had been using a VC709 for a long time and are now trying
    to switch to the FLX712.

    We use a Dell PC with 24G of RAM.  We have already understood
    that this computer will not support I2C to the PCIe slot, thus
    we can't use fflash for configuring the FPGA.  But we have had
    no problem (and are fine with) using vivado and a JTAG adapater
    to configure the FPGA.

    The old version of the driver we have been using with our FLX709
    was struggling with 6 GB, but working well with 12GB of system RAM.
    The current version of the driver with the FLX712 was clearly
    not happy with only 12G but seemed more pleased with 24G.
    However it fails with "This version of the driver is for regmap 4.0".


    We are using this FW

    > 
https://atlas-project-felix.web.cern.ch/atlas-project-felix/user/dist/firmware/4.8/
    > 
FLX712_GBT_24CH_CLKSELECT_GIT_ReleaseCandidate-rm4.8_rm-4.8_46_191022_11_02.tar.gz
  14-Nov-2019 14:40   56M

    And we installed this driver

    > 
https://atlas-project-felix.web.cern.ch/atlas-project-felix/user/dist/software/driver/
    > tdaq_sw_for_Flx-4.1.0-2dkms.noarch.rpm  12-Nov-2019 12:58   59K for 
felix-4.0.9

    All this is on CentOS7

    > [root@msutcc ~]# cat /etc/redhat-release CentOS Linux release 7.8.2003 
(Core)

    It looks like the driver is not able to find the FW

    > [root@msutcc ~]# service drivers_flx status
    > cmem_rcc             1096412  0 
    > 
    >>>>>>> Status of the cmem_rcc driver
    > 
    > 
    > CMEM RCC driver (nopage) for release tdaq710_for_felix_4.1.0 (based on 
tag ROSRCDdrivers-00-01-00)
    > 
    > The driver was loaded with these parameters:
    > gfpbpa_size = 8192
    > gfpbpa_quantum = 4
    > gfpbpa_zone = 0
    > 
    > __get_free_pages
    >   PID | Handle |         Phys. address |               Size | Locked | 
Order| Name
    > 
    > GFPBPA (size = 8192 MB, base = 0x00000002ce800000)
    >   PID | Handle |         Phys. address |               Size | Locked | 
Name
    >  
    > The command 'echo <action> > /proc/cmem_rcc', executed as root,
    > allows you to interact with the driver. Possible actions are:
    > debug    -> enable debugging
    > nodebug  -> disable debugging
    > elog     -> Log errors to /var/log/messages
    > noelog   -> Do not log errors to /var/log/messages
    > freelock -> release all locked segments
    > io_rcc                 21598  0 
    > 
    >>>>>>> Status of the io_rcc driver
    > 
    > IO RCC driver for release tdaq710_for_felix_4.1.0 (based on tag 
ROSRCDdrivers-00-01-00)
    > Dumping table of linked devices
    > Handle | Vendor ID | Device ID | Occurrence | Process ID
    >  
    > The command 'echo <action> > /proc/io_rcc', executed as root,
    > allows you to interact with the driver. Possible actions are:
    > debug   -> enable debugging
    > nodebug -> disable debugging
    > elog    -> Log errors to /var/log/messages
    > noelog  -> Do not log errors to /var/log/messages
    > Current values of the parameter(s)
    > debug    = 1
    > errorlog = 1
    > flx                    43349  0 
    > 
    >>>>>>> Status of the flx driver 
    > 
    > FLX driver 4.1.0 for RM4 F/W and TDAQ release tdaq710_for_felix_4.1.0. 
Distributed with driver RPM 4.1.0
    > 
    > Debug                       = 0
    > Number of cards detected    = 2
    > 
    > 
    > Locked resources
    >         card | global_locks
    > =============|=============
    >            0 |   0x00000000
    >            1 |   0x00000000
    > 
    > Locked resources
    > card | resource bit |     PID |  tag
    > =====|==============|=========|=====
    > 
    > Error: Card 0 does not have the required F/W. The regmap register 
contains 0xffffffff
    > 
    > Error: This version of the driver is for regmap 4.0
    > 
    > Error: Card 1 does not have the required F/W. The regmap register 
contains 0xffffffff
    > 
    > Error: This version of the driver is for regmap 4.0
    >  
    > The command 'echo <action> > /proc/flx', executed as root,
    > allows you to interact with the driver. Possible actions are:
    > debug     -> Enable debugging
    > nodebug   -> Disable debugging
    > elog      -> Log errors to /var/log/message
    > noelog    -> Do not log errors to /var/log/message
    > rm3       -> Enable compatibility with RM3 F/W
    > rm4       -> Disable compatibility with RM3 F/W
    > swap      -> Enable automatic swapping of 0x7038 / 0x7039 and 0x427 / 
0x428
    > noswap    -> Disable automatic swapping of 0x7038 / 0x7039 and 0x427 / 
0x428
    > clearlock -> Clear all lock bits (Attention: Close processes that hold 
lock bits before you do this)


    We are guessing the regmap version is not the true problem
    ... or did we miss something here?

    Adding the debug option shows this in /var/log/messages

    > Nov  1 18:50:09 msutcc kernel: cmem_rcc(cmem_rcc_init): Failed to 
determine the BPA base address
    > Nov  1 18:50:09 msutcc kernel: cmem_rcc(cmem_rcc_init): Check if BPA 
memory has been reserved at boot time
    > Nov  1 18:50:09 msutcc kernel: flx(flx_Probe): Failed to map MSI-X BAR 
for card 1
    > Nov  1 18:50:09 msutcc kernel: flx(flx_Probe): Failed to map MSI-X BAR 
for card 0
    > Nov  1 18:50:09 msutcc kernel: flx_init: FLX driver loaded, found 2 
device(s)


    Is this hinting at an issue with contiguous memory allocation at boot?
    If so, is there any way to reduce the demand for a case
    that will not involve readout and only use 2 channels?

    Thanks,
    Philippe