From: Jennifer Cline

Sent: Wed Jun 03 16:14:20 2015

To: Troy Collison

Cc: Dan Huff; Phillip Bender; Jason Clifford; Fasth, Bill

Subject: RE: WWTP SCADA

Importance: Normal

Attachments: image001.jpg; may 9 - may 11 alarm list.txt; May 9 inf flow.pdf; wp_lmi_security.pdf;

 

Hi Troy,

Thank you for your prompt response, I have a few additional questions.

The alarms on Friday 5/9 were because the operators cleaned out the wet well of the TPS as a regular maintenance practice. The pumps were in operation and functioning when the crew left on Friday. Also seen to be functioning fine when Jake was at the plant on Saturday morning.

Regarding the diagram, it only appears to be from Saturday 5:53 am to Sunday 5:52 am. The graph also indicates the pumps may have been operating on Sunday around noon when the flows dropped to normal again. Can you provide a graph from Sunday 5:00 am to Monday at 10 am as well? Is it possible to snip copy of the screen and send as a .pdf?

I spoke with Jake today and he confirmed the HWS by-pass alarms were enabled during this event. Is there anything else that would have stopped the alarm from communicating to the dialer?

Regards,

Jennifer Cline, P.E.

City of Molalla

Public Works Director

O: 503.759.0218

F: 503.829.3676

image "file:///C:

From: Troy Collison [mailto:tbc@industrialsystems-inc.com]

Sent: Wednesday, June 03, 2015 3:06 PM

To: Jennifer Cline

Cc: Dan Huff; Phillip Bender

Subject: WWTP SCADA

Jennifer

During today’s discussion I was asked to look into several things:

1) Availability of historical alarm data (older than 24 to 36 hours)

a. This is available. Please see attached text file covering the period of May 9th – 11th

i. Several Transfer Pump Station alarms occurred at 3:55 PM on Friday the 8th

1. VFD 2 Fault

2. VFD 3 fault

3. General alarm

ii. At 4:02 PM Transfer Pump 1 failed on low flow and then at 4:05 – 4:08 the VFD on this pump issued several other alarms

iii. At 4:12 PM, the lead pump selection was changed several times

iv. At 4:18 PM, the on/off levels for the pumps were changed

v. The next things logged, other than report generation, is the loss of access to the SQL database at 1 second after midnight on the 10th. (this is NOT the com fail issue we have been examining and does not affect the operation of the plant)

vi. Then the HWS bypass alarm at 8:39 AM on May 10th

2) Can we tell from any historical data, when the high level occurred

a. Yes, please see attached plot of the influent flow. This shows that the influent parshal flume began experiencing an abnormal high level at about 3:00 PM on Saturday May 9th, 2015

b. NOTE - I outlined the flow in pen before scanning this print out to enable a person to read it. The actual print out is difficult to see but is readable. The scan would not be. Viewing the data on the SCADA computer is, of course, very clear

3) When was “Log Me in” installed at the WWTP?

a. Friday May 15th, 2015

4) How good is the security rating of “Log me in”? It is as described during today’s meeting. This is outlined on the webpage at this link.

a. https://secure.logmein.com/products/pro/security.aspx

b. I did not find a review of this product by any governmental entities

c. I also attached a white paper on the security of this product

i. I believe it is safe to say, the software provides a reasonably secure remote connection:

1. It is protected by two sets of user names and passwords

2. The software CAN be configured to store logon and logoff events

3. Data being transmitted over the internet is encrypted

4. The software is configured to detect when someone tries to intrude on the link

5) What triggers the HWS bypass alarm?

a. This is triggered by a discrete input to the Programmable Logic Controller (PLC). In one place in the PLC logic it is listed a afloat switch and in another it is listed as an alarm output from the HydroRanger level transducer so I am not sure which is accurate

I hope this information is helpful. Please do not hesitate to call with any questions,

Best Regards,

Troy B. Collison

image

 

5835 NE 122nd Avenue, Suite #110

Portland, Oregon 97230

http://www.industrialsystems-inc.com

Phone: 503.262.0367

Cell: 360.957.5039

Fax: 503.262.0371

CCB # 196597 WA #INDUSSI880K9

AK# 1018436