The Security Practice

Socket Capable Browser Plugins Result In Transparent Proxy Abuse

We're pleased to announce the availability of 'Socket Capable Browser Plugins Result In Transparent Proxy Abuse'. This document outlines the abuse case in CERT's VU #435052 advisory published last month.

Abstract

"Transparent proxies allow organizations to influence and monitor the traffic from its users without their knowledge or participation. Transparent proxies act as intermediaries between a user and end destination, and aren't generally apparent to users sitting behind them. Enterprises, Hotels, and Internet Service Providers often use transparent proxy products to lower bandwidth consumption,speed up page loads for their users, and for monitoring and filtering of web surfing. When certain transparent proxy architectures are in use an attacker can achieve a partial Same Origin Policy Bypass resulting in access to any host reachable by the proxy via the use of client plug-in technologies (such as Flash, Applets, etc) with socket capabilities. This write up will describe this architecture, how it may be abused by Flash, its existence in various network layouts, and mitigations."

Download Paper: http://www.thesecuritypractice.com/the_security_practice/TransparentProxyAbuse.pdf

CERT Advisory: http://www.kb.cert.org/vuls/id/435052

PayPal's Information Risk Management Team is Hiring

We're hiring two application and project security consultants and a security manager.   Duties include security analysis of projects, high level risk analysis, threat modeling, implementation guidance, deployment guidance, and some strategic consulting and research.

Here are some easily clickable links:

Manager, Information Security - Phoenix
Principal Information Security Engineer - Phoenix
Principal Information Security Engineer - San Jose

You can also check out the PayPal jobs board at https://www.paypal.com/jobs and search for security jobs.

Position on DNSSEC and Root Zone Signing

Hello, Andy Steingruebl here.

On October the National Telecommunications and Information Administration (NTIA) published a Notice on Inquiry in the Federal Register related to DNSSEC and Root Zone Signing.

PayPal has responded with comments, which are posted below and should show up soon at the NTIA DNSSEC site.

Our submission is below.

**************************************************************

TO:                  DNSSEC@ntia.doc.gov

FROM:            Michael Barrett, Chief Information Security Officer, PayPal

RE:                Notice of Inquiry on Enhancing the Security and Stability of the Internet’s Domain Names and Addressing System — 73 Fed. Reg. 59608 (Oct. 9, 2008) and http://www.ntia.doc.gov/frnotices/2008/FR_DNSSEC_081009.pdf

DATE:             November 21, 2008

Thank you for this opportunity to comment on implementation of DNSSEC (Domain Name and Addressing System Security Extensions) at the root zone level, on behalf of PayPal.  Since 1998, PayPal has been the global leader in online payment solutions.  While PayPal has only been in business a decade, it has about 165 million users globally, operates in 190 markets, and settles in 19 major currencies; in 2007 we processed over $47 billion in payments.

As a major online financial services institution, the safe and secure operation of the Internet is of vital importance to PayPal’s business.  This safety and security, and indeed the ongoing viability of electronic commerce, depend on the trust of consumers.  They must know that they are interacting with the real PayPal and not a phishing site.  PayPal has been a leader in the deployment of technologies that enable this trust.  PayPal was one of the first deployers of EV (Extended Validation) SSL (Secure Sockets Layer) digital certificates.  It applies digital signatures to all email it sends to consumers.  These techniques enable consumers, ISPs, and software clients to make trust decisions about what content is legitimately from PayPal.  However, there are limits to what any single company can do to make the Internet experience of our customers safe and secure.  The current weakest link in the trust chain on the Internet is the DNS (Domain Name System).

In this context, the rapid adoption of DNSSEC (DNS Security Extensions) and signing of the root zone is an urgent requirement.  We applaud NTIA for initiating this inquiry, and urge it to move with all possible speed to implement DNSSEC.  Inaction or further delay would be detrimental to the interests of consumers and other Internet users, and to the healthy growth of electronic commerce.

Background

We believe that there are three major threats to the internet user that are related to authentication or identifying who users are communicating with:

1. Spoofed websites
2. Spoofed Email
3. Spoofed DNS

The first two threats are being managed.  SSL, along with EV certificates, largely solves the general problem of spoofed websites from a technology standpoint - at least for those sites that can put all of their traffic across SSL.  Email signing via DKIM (Domain Keys Identified Mail) along with other technologies such as ADSP (Author Domain Signing Practices), and reputation services, mostly solve the problem of spoofed email.  This leaves DNS spoofing as the main unaddressed vulnerability in this sphere.  DNSSEC is the best available tool for addressing this threat.

DNS spoofing currently falls into two categories, both of which we believe are threats to PayPal, to our customers, and to consumers and Internet users in general.  The first type of spoofing is done by an attacker.  These attacks can occur on public networks such as a public wireless access point, or they can be targeted against DNS infrastructure at the ISP.  With the vulnerabilities inherent in the DNS protocol’s UDP delivery mechanism and lack of strong encryption for transport, DNSSEC represents the only way for a consumer and an ISP to identify a legitimate DNS response.

The second threat of DNS spoofing comes from DNS infrastructure operators themselves.  Many DNS providers, registries, and ISPs are performing DNS spoofing as a way to monetize user errors, such as when a user attempts to access a domain name that does not exist.   In the case of DNS spoofing of non-existent domains we believe this is an internet governance problem between ICANN and the registry for a given TLD.

A more insidious type of DNS spoofing has taken hold lately that we believe is also a threat.  This is the spoofing of DNS responses for non-existent sub-domains of legitimate domains within DNS.  Many ISPs in the US and elsewhere are spoofing DNS replies for DNS requests that generate an NXDOMAIN record type from the SOA (Start of Authority) for that domain.  For example, when a user mistypes ww.paypal.com (intending to type www.paypal.com), the ISP will serve that user a spoofed DNS reply that points the user at an ad server run by the ISP.  This ISP ad server will receive cookies intended for the domain in question, can set cookies for the domain in question, and is now within the security perimeter of the domain owner without the owner’s permission.  As a result, security features within the intended site may no longer work properly, and the consumer’s security is jeopardized.  In the worst-case scenario, bad actors could use this technique to generate a  spoofed reply pointing to a fake site intended to mimic the actual PayPal site.  Consumers using an ordinary internet connection have no mechanism for determining that this response was spoofed.  DNSSEC is currently the only security technology that would reveal to the typical consumer that the response was spoofed, and allow the consumer to make an informed choice about this illegitimate behavior by the ISP. 

The Value of DNSSEC

DNSSEC and root zone signing solve several critical security problems and provide the basis  for new developments in the secure delivery of information, such as signed security policies, secure SPF policy delivery, and the secure delivery of information such as that contained in DNS SRV records.  No other alternative for providing the secure delivery of this information is on the horizon.  Kaminsky’s work on exposing new exploits against previously known structural weaknesses in the DNS protocols has increased the pressure and need for DNS authentication.  While theoretically, there might be better technical solutions than DNSSEC, we do not believe that there is enough time to contemplate development of an alternative protocol to DNSSEC, which currently represents the only viable option for the secure delivery of this type of information. 

The main hurdles to overcome are those related to responsibility and authority for management of the root zone signing keys and the signing itself.  In this matter we believe there are short term and long term considerations. 

In the short term we believe that getting the root zone signed, even under the auspices of a single root signing authority, is preferable to waiting for consensus to emerge about per-country or M-of-N signature proposals.  While we recognize that single authority systems may not be as robust in the face of certain types of attacks, and could be vulnerable to political pressure, we believe that these issues should be decoupled from the implementation timeframe for initial signing of the root zone.  A single authority should sign the root zone as soon as possible.  We believe that there is now limited time before usable brute force attacks are available to criminals, and therefore rapid deployment of DNSSEC is the only viable solution.

Based on currently submitted proposals we believe that IANA or ICANN should be given the authority and responsibility for implementing the initial root zone signing.  Of the choices presented in the Notice of Inquiry, Proposed Process Flow #4 would accomplish this, though there are other options for achieving this result.  Given the current role of ICANN in governance, and of IANA (as administered by ICANN) in root zone maintenance, this would be the natural choice for the authority and responsibility for root zone signing.  However, we do have operational concerns related to security practices of whoever is chosen.  On this issue, we urge NTIA to seek expert guidance from organizations (such as root Certification Authorities) that have experience in handling the type of sensitive materials involved in these operations.   

Longer term, we believe that some form of M-of-N or simple multiple signing authorities approaches should be explored.  Vesting control of root zone signing in one entity is too tempting an attack target and too subject to political pressure from a single government or entity.  Ultimately end-user and DNS infrastructure software will be the main consumers of DNSSEC data.  Just like the current situation in web browsers, where a list of Root CA Certificates is deployed, we expect a similar model will be necessary for DNSSEC.  Given the already existing software infrastructure and ecosystem for allowing multiple Root CA certificates in end-user software, we believe M-of-N, or at the very least multiple signatures on the root zone or other TLDs, is in practice workable.  We recommend that this option be further developed, but we do not think it would be prudent to wait until it is fully developed before signing the root using DNSSEC. 

Finally, I would like to publicly recognize the significant contribution of Andy Steingruebl, Principal Information Security Engineer in the PayPal Information Security team, in helping form our opinion.  He facilitated my team in coming to a consensus on this difficult issue, and performed extensive analysis of the various options presented in the Notice of Inquiry.

PayPal appreciates the opportunity to provide comments.  We would be glad to provide further information on this topic and look forward to working with NTIA in closing the existing vulnerabilities of the DNS through prompt adoption of DNSSEC.  

Respectfully submitted,

Michael Barrett
Chief Information Security Officer, PayPal

Firefox 3.0 and self-signed certificates

Hello, Michael Barrett here.

I read a rather worrying criticism of Firefox 3.0 on “Risks” the other day, which made me realize that perhaps there isn’t a common agreement amongst the infosec industry about the threatscape and how we should prioritize our response to them. Specifically, the complaint against Firefox 3.0 is that the user experience has been deliberately crafted to make it hard to accept self-signed certificates. The argument is that there are times when simply establishing an encrypted tunnel (i.e. an SSL session) is all that’s needed.

 

I certainly wouldn’t argue that encryption is unnecessary, just that the threat has changed.  While our old “friend” Mallory isn’t particularly busy these days, it’s pretty clear that he’d be having a field day if he could easily penetrate communications across the Internet. The attacker however is no longer limited to passive eavesdropping. Modern attacks use active DNS spoofing, active MITM attacks and the like, on public networks.  The main threats these days are against the weakest link in the chain – the end user. That’s why phishing is such a popular method of e-crime – it’s simple and it works. It relies completely on the gullibility of users in clicking on links in e-mails apparently from organizations with whom they have a relationship.

 

However, it’s equally clear that almost everyone who wants to communicate securely using a browser can afford an SSL certificate from CAs such as GoDaddy, Thawte, etc. The cost of single certificates from these sources can only be described as nominal.

 

My company is a major target of phishing, and as such we’ve spent quite a bit of time researching what anti-phishing approaches work We published a whitepaper on this topic (which can be found on the company blog at www.thepaypalblog.com), which explains this in detail. However, a couple of relevant conclusions are that: 1) the vast majority of users simply want to be protected, 2) there’s no single “silver bullet”, and 3) that what we describe as “safer browsers” such as IE 7, and Firefox 3.0 are a significant part of the solution based on their improvements in user visible security indicators and secure-by-default behaviors.

 

I conflated two or three separate ideas in that last sentence, and I should explain them. The general logic is that most users should never be presented with a security dialog that gives them a choice – if they are, there’s typically at least a 50:50 chance that the wrong decision will be made. Instead, the browser should make the decision for them. However, in the case of self-signed certificates it’s almost impossible to see how any technology can disambiguate between legitimate uses and criminal ones.

 

When viewed through this lens, the changes to the Firefox user experience for self-signed certificates makes perfect sense. It’s not that self-signed certificates are impossible to use – but for most users, the experience will be such that they won’t accept them. In the unsafe world in which we live, that will be the right choice. For organizations which wish to use self-signed certs internally, it is still technically possible – but it will require either explicit user training, or deployment of pre-installed certificates on PCs.

 

I should also add that the major security features which have been added into the most recent browser versions (and which we believe are necessary in order to be considered ‘safer’) are exactly those which impact this area. That is: support for Extended Validation certificates, which make it clear to end users whose web site they’re on; and support for spoof-site black lists, so that users can’t easily reach spoof-sites.

 

While I’m personally a great supporter of the “Risks” list, I think it’s important that the infosec industry speaks with good consensus about risks. In this case, I believe that the criticism of Firefox 3.0 was simply misguided and ill-informed. This is not helpful.

 

Communicating a Site Security Policy

Hello, Andy Steingruebl here.

As a security team we find that we have several constituencies all of whom require a different form of communication.  In general we spend most of our time communicating internally with project managers, product managers, developers, and operations staff.  We generally know how to reach out to these folks and our main formal communication vehicle is the policy or standard.

Sometimes we're called upon to communicate directly with our users.  Usually this takes the form of a blog posting on the PayPal blog, or we review marketing materials, or even the information that goes into the PayPal Security Center.

One of the groups that is hard to reach however is the technically savvy user.  We can't share our internal policies and standards, but the information in our security center isn't really targeted towards this type of user either.  Recently I was reading a post to Risks Digest wherein there was a comment/complaint about a Treasury Department website and their web security policies/practices. 

I spoke with the poster and we came to the conclusion that:

1. The Treasury Department had been using a different threat model than he had when analysing their security measures.
2. the Treasury Department had no way of communicating their threat model and security policy to the end user of their service.

As a result of these two situations, the user was in a position of believing that the Treasury Department had a faulty security model, when in fact my guess is that their model is actually entirely reasonable and just designed to protect against a different threat.

If a site like PayPal wants to reach out to a broad audience, and clear up some of the doubts, misconceptions, and so on that arise over time, we need to find a way to safely but openly discuss our security policy assumptions with our end users.  We need to explain that we believe Phishing, Malware, and Operating System weaknesses are some of the issues we're protecting against, or that we believe are our biggest threats.

We don't want everyone to need to sort through these sorts of messages, but we think it might be useful to have some form of communication on the site about these policies. 

In discussing this with the original Risks poster, he suggested that in their case perhaps a simply statement such as "This special method of entering your password is designed to protect against the possibility that malicious software is installed on your computer and eavesdropping on your
keystrokes." 

I like this element of simplicity, and in the future I'll be trying to craft exactly these sorts of statements for a broader audience.

Disclosure policies – what constitutes “responsible” disclosure, vs irresponsible disclosure?

Hello, Michael Barrett here.

Anyone who works in the information security business would have to have been in full Rip Van Winkle mode to have missed the debate about disclosure policies over the last several years.  The agonizing that we have collectively done over what constitutes “responsible” disclosure would have been greatly admired by a mediaeval philosopher, more accustomed to debating how many Angels can dance on the head of a pin.

2007 marked something of a watershed in that the practice – which had previously been kept pretty quiet – of companies, such as iDefense, purchasing vulnerabilities on the black market turned into at least a publicly visible grey market.  While there was some debate about it, I was personally surprised by how little the ethics of this were discussed.

2007 also marked something of a change in that it was the first year wherein e-commerce websites, such as PayPal, started publishing clear policies as to what they consider to be responsible.  We published our own in late 2007, and it can be found at:

https://www.paypal.com/us/cgi-bin/webscr?cmd=xpt/cps/securitycenter/general/ReportingSecurityIssues-outside

While I wouldn’t necessarily describe our own policy as perfect, a great deal of work went into it.  Actually, considering how short the policy is, that statement may seem surprising, but nonetheless it is the case.

We’re very interested in hearing feedback from the community as to whether we’ve got this policy right, or close to it.  We’re certainly prepared to tweak it, if we hear convincing arguments that there are good reasons for doing so.  There are a couple of areas of our policy where we agonized for some time.

We frankly fudged over the length of time we have to ensure that a particular vulnerability is fixed.  I was personally in favour of describing the maximum period that we’d commit to; however, the team more or less unanimously over-rode me on that one.  In the end, we settled on the weasel words “reasonable time” instead.  What convinced me was that:

1) In order to give a publicly useful time commitment, we would have to explain our entire vulnerability classification methodology (which would be several times longer than the policy itself), because what constitutes “reasonable” isn’t basically absolute but is instead situational.  For example, in the case of an extremely serious vulnerability (an SQL injection flaw, for example), we’d almost certainly respond with extreme speed, taking the offending page down, even if there was non-trivial business impact.  However, an extremely difficult-to-exploit and scope limited XSS, we would probably fix in a more leisurely fashion – especially if it’s one where we can easily implement automated monitoring to detect if it ever gets exploited.  (By the way, this is one of the key differences in vulnerabilities in websites vs. vulnerabilities in distributed software – by definition, we have much more immediate control over how we respond.)

2) Having said that, we also don’t necessarily have direct control over our development and functionality rollout priorities.  (Shocking, eh?  Who knew that Information Security isn’t the Centre of the Universe?)  Imagine a case for a low priority issue, where we were doing automated monitoring.  Of course, we’d still have it in the priority queue for fixing but we’d frankly not necessarily fight too hard if our dev teams de-prioritized the fix over more important business enhancements – especially if there was no sign of any exploit on the horizon.

While I grudgingly accepted all of these arguments, my concern was and is that “reasonable” is in the eye of the beholder.  While we try to be responsible and prudent, I can certainly imagine other organizations whose functional definition of reasonable might translate to “when Hell freezes over”.  I’d rather be able to craft a policy that, if adopted by one of those companies, would effectively change their behaviour.  Alas, in that regard, I don’t believe we’ve succeeded yet.  The obvious question is whether it’s actually possible to come up with a short pithy policy that also meets that particular litmus test.  I’m not sure that it is.

More on disclosure in a few days, where I’ll talk about the really contentious aspects…

Welcome and a brief orientation

The authors of posts in this blog come from the Information Risk Management at PayPal. As a group we are interested in discussing what we see happening in the world of IT security as it relates to PayPal. So topics you might expect to see covered include trends that we are seeing in our security practice, architectural issues of various types, comments on the safety or effectiveness of protocols, operating systems or browsers.

This is not a marketing venue (as you will no doubt discover from the degree of polish on the posts) and in fact it s not an official PayPal blog at all.

We welcome dialog with other security professionals about the issues raised here - polite disagreement and debate is welcome.

Informed questions are welcome and we will answer what we can within the bounds of good security practise, but if you have a complaint about PayPal, please raise it through one of the many formal channels that exist (or feel free to express your opinion on any of the Internet sites dedicated to that purpose).

Owing to the obvious challenge of keeping the signal to noise ratio high, comments will be moderated.

The opinions expressed are those of the individual authors.