Optical Multiplexer Board 9U Home Page 




Optical Multiplexer Board 9U


TileCal  is the hadronic tile calorimeter of the ATLAS-LHC experiment and consists in terms of electronic readout of roughly 10000 channels read each 25 ns. Data gathered from these channels are digitized and transmitted to the Data Acquisition System (DAQ) following a three level trigger system (Fig.1).

Fig. 1.        The ATLAS three levels trigger system.


The main component of the back-end electronics of the TileCal sub-detector is the Read-Out Driver (ROD) which is placed between the first and the second level trigger. The ROD has to pre-process and gather data coming from the Front End Boards (FEB) and send these data to the Read-Out Buffers (ROB) in the second level trigger.

            TileCal electronics will receive about 2 Gy/year (0.2 Krad/year) of radiation for a total dose of 20 Gy in the experiment lifetime  To measure radiation hardness of TileCal FrontEnd electronics, tests were conducted with proton beams in different areas and with different beam sizes. Thanks to these tests, three non-destructive kinds of errors were found:

·        Transient error in the data flow out to the ROD.

·        Permanent errors in the data flow requiring reset.

·        Latch-up error with an increment in current consumption of 60 mA.

To reduce data loss due to radiation effects, the TileCal collaboration decided to include data redundancy in the output links of the FrontEnd. This was accomplished using two optical fibres which transmit the same data. At ROD system level, data redundancy is used to discard the fibre with errors due to radiation. The checking is based on rightness of the Cyclic Redundancy Codes (CRC) of the data packets on both fibres. This is also necessary as the ROD motherboard is expecting just one fibre per channel. For this purpose a new module, called PreROD or Optical Multiplexer Board (OMB) was conceived (Fig. 2). This board would be able to provide, in case of error in one link, the correct data to the ROD input by analyzing the Cyclic Redundancy Codes (CRC) of the data packets on both fibers coming from the FEB.


Fig. 2.        The OMB in the TileCal data acquisition chain.


The final OMB 9U prototype (Fig. 6) is designed from the 6U board experience. This prototype is conceived in a 1 to 1 ratio with respect to the RODs, i.e. each board has 16 input links and 8 output links, setting the final format in a 9U VME standard board.


Fig. 3.        Picture of the OMB 9U prototype board.

From the functionality point of view there are some minor modifications being the most important, the inclusion of the TTC receiver chip. This would lead to the possibility receiving the trigger directly from the TTC system. In view of future upgrades and to increase the functionality the design includes four PMC connectors for mezzanine boards connected to the CRC FPGAs and is designed for 80 MHz operating frequency instead of the nominal 40 MHz of the LHC. This last issue poses some problems related to signal integrity and component placement aspects. Among them, the use of a single JTAG chain for the programming of all the FPGA chips in the board, the bus connecting the VME controller and the CRC controllers and the clock distribution are the main concerns.

The PCB Specifications

The OMB final prototype layout is a 10 layer PCB which optimize the cross-section to minimize signal integrity problems. Fig. 7 shows the arrangement of the layers. We tried to keep every signal layer between two power planes or, when it was not possible, routing the two adjacent layers orthogonally.


Fig. 4.        OMB 9U prototype layer stackup.

The power distribution is also a concern in this board as we must supply several voltages. All the FPGAs need 3.3 V for the I/O and 1.5 V for the internal operations. The NIM to TTL conversion for the external trigger signals needs a 12V supplied voltage while other logic circuitry needs 5 V. The 12 V and 5 V power supplies are taken from VME bus or, when it is not available or for testing, from special pins on the board. The generation of the lower voltages (3.3 V and 1.5 V) is accomplish by voltage regulation from the 5 V main power supply. With this configuration, the power plane in layer number 2 is connected entirely to 3.3 V whereas the power plane in layer number 9 is a split plane with 1.5 islands below the FPGAs (Fig. 8).


Fig. 5.        Power distribution in the internal layer number 9.


Fig. 6 shows the top layer layout with the main components highlighted. In the OMB 9U board there are more than 1200 components connected with more than 2000 nets. The components distribution is not uniform and they are mainly placed near the front-panel since these components are used to process or to inject data through the optical connector placed in the front-panel. The mezzanine connectors for the daughterboard cards are mounted in the center of the board. Finally, the VME interface and the TTC receiver are placed close to the VME connectors.

The Optical Connectors

Stratos Lightwave dual optical receiver (M2R-25-4-1-TL) and transmitter (M2T-25-4-1-L) are chosen to optimize the space in the board. Since 16 inputs and 8 output links are needed there are 8 dual receivers and 4 dual transmitters in each board. The dual receiver connectors receive an optical fiber from the front-end and transform it into a electrical PECL differential signal whereas the dual transmitter connectors transmit the differential signal to the optical fiber. These differential lines, which connect the optical connectors and the G-Link chips, were manually routed and their impedance controlled, because these lines transmit high speed signals (640 Mbps).

The G-Link Chips

The G-Link chips serialize (HDMP-1032) and de-serialize (HDMP-1034) the data transmitted and received through the optical connectors. The HDMP-1034 receiver chip receives the differential signal directly from the optical receiver and transforms it into a 16-bits bus. These chips are individually clocked with a 40 MHz oscillator placed close to the chip. Moreover, the HDMP-1032 transmitter chip receives the 16-bits bus from the CRC-FPGA and transforms it into a PECL differential bus. These chips are also clocked at 40 MHz but this clock is generated internally by the FPGA firmware. There are in total 16 receiver chips and 8 transmitters.


The CRC-FPGA is the main component of the OMB 9U because they are responsible for the data checking in the CRC operation mode and the generation of data in the injection mode. There are 8 CRC-FPGAs in each board and they are ALTERA EP1C12 devices, also used in the previous 6U prototype design. These devices will receive directly the front-end data for the CRC checking. Nevertheless, it is possible to include more functionalities as a Bunch Crossing Identification (BCID) checking, because these devices will receive through the TTCrx chip all the TTC information generated by the Central Trigger Processor (CTP). All the error counters as well as the configuration and status registers are also included in the CRC-FPGAs firmware and they are readable and/or writable through the VME bus.

Besides, the CRC-FPGAs are connected to the Processing Units (PU) connectors for future upgrades. In this case, the data received in the CRC-FPGA might be sent to the PU for processing tasks before its transmission to the RODs.

Finally, the CRC-FPGAs firmware is downloaded by using the JTAG chain or in a Passive Serial mode by using the Erasable Programmable Read-Only Memory (EPROM) memories mounted in the board, there are two EPROM for each 4 CRC-FPGAs.


Fig. 6.        The OMB 9U prototype main components.

The VME Interface

The interface with the VME bus is managed by the VME-FPGA, which is implemented in a CYCLONE EP1C20 device. The VME-FPGA generates the geographical address of the board and represents the interface between the VME bus and the CRC-FPGAs in order to read and/or write the registers physically placed in the CRC-FPGAs. It provides also the VME communication with the TTC-FPGA. Besides, the VME-FPGA might be used to internally generate a trigger signal in the injection mode.

The TTC Interface

The TTC interface is implemented in the OMB 9U with a TTC receiver chip (TTCrx) and the TTC-FPGA. The TTC information is received in the TTCrx through the backplane and it includes the Bunch Crossing (40 MHz), the Bunch Crossing Reset (BCR), the Level 1 Accept (L1A), the Event Counter Reset (ECR) and the Trigger Type (TType). With these signals, the TTC-FPGA generates the Bunch Crossing Identification (BCID) and the Event Identification (EVID). These signals and the TType are transmitted to each CRC-FPGA with each L1A received.

The TTC information might be used in the OMB 9U board to check the BCID of the data received from the Front-End or to inject data to the ROD with actual TTC information.


Operation Modes

The CRC Checking

In the CRC checking operation mode the OMB 9U receives 16 fibers from 8 Optical Interface Boards (OIB) and transmits 8 fibers to one ROD (Fig. 7). Each CRC-FPGA must check the CRC of two redundant inputs and decide which one is transmitted to the ROD system. Moreover, the decision has to be taken in real time but a latency time is introduced in the acquisition chain. The algorithm that decides which fiber is transmitted and which is discarded consists of two simultaneous operations. The events received through each input link are stored in two different memories while the CRC is computed. The last word of the event includes the global CRC computed in the Front-End over the entire event. When this word is received the algorithm checks the CRC and decides which memory output is enabled.


Fig. 7.        Dataflow of the OMB 9U prototype CRC checking operation mode.

Apart of the global CRC, it is possible to decide which fiber is transmitted to the ROD system by checking the DMU CRC (included in the trailer) or the BCID (included in the header) of each DMU data block (16 per event). All the errors detected are counted and stored in the corresponding internal register.

The Injection Mode

There are two different injection modes as described above for the previous 6U board: the counter and the memory injection modes. The main differences with the previous 6U version are the number of output channels and the possibility of the injection of data with actual TTC information. With a OMB 9U it is possible to inject data to a whole ROD through its 8 optical outputs (Fig. 8). Furthermore, the TTC feature permits the injection of data with the TTC information received through the backplane. Since this information is also received in the ROD, it is possible to test the TTC synchronization at ROD level with the data generated in the OMB 9U.


Fig. 8.        Dataflow of the OMB 9U prototype injection mode.

The complete design including schematics and PCB artwork is available in the CERN EDMS web page. (EDA-01518-V1-0)


Last updated 18 may 2001

Maintained by Juan Valls & Alberto Valero © IFIC, Universitat de València